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CHAPTER  1 
Introduction 

Artificial  Neural  Networks  (ANN’s)  are  computing  networks  that  are  adaptable  or 
trainable,  and  naturally  parallel.  ANN's  are  also  referred  to  as  parallel 
distributed  processors,  adaptive  systems,  connectionist  models,  self-organizing 
systems,  and  neuromorphlc  systems  [1,2].  ANN’s  are  biologically  inspired  and  are 
based  on  neurological  models[l  ,2,3,5, 8].  Neural  networks  offer  significant  potential 
benefits  in  the  areas  of  pattern  and  speech  recognition,  information  processing, 
nonlinear  controls,  and  expert  systems[2,3]. 

Currently,  there  is  a  demand  for  high  speed,  low  cost,  and  small  size  ANN’s  to  serve 

f  •  1  ' ! 

a  variety  of  applications.  Silicon  large  scale  neuro  chips  have  been  designed  and 
demonstrated  [51,52,53,54,55,56].  However  these  are  limited  in  interconnect  density. 
Ozaktkas  and  Goodman  [57  ]  point  out  that  low  interconnect  density  arises  from  large 
area  requirements  of  conductor  guided  interconnections. 

To  overcome  the  problems  associated  with  electronic  ANN’s,  various  optical  neural 
network  configurations  have  been  investigated  [34  through  50].  Optics  offer  many 
advantages  such  as  massive  parallelism,  free  space  interconnects,  and  immunity  to  EMI 
noise.  However,  most  of  these  networks  are  impractical  and  only  demonstrable  in  a 
laboratory  environment.  By  combining  optics  and  electronics,  optoelectronic  neuro¬ 
chips  offer  an  alternative  to  previous  networks  [59].  A  hybrid  optoelectronic  neural 
network  for  recognizing  simple  characters  has  been  designed  and  developed. 
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This  thesis  provides  a  tutoriai  in  the  areas  of  neural  networks  and  related 
optics  and  reports  on  the  development  of  an  optoelectronic  artificial  neural  network. 
Chapter  2  covers  some  important  foundations,  and  provides  a  general  framework  for 
neural  networks.  Chapter  3  discusses  the  adaptive  linear  combiner  or  ADALINE  neuron 
and  linear  separability  which  provides  a  justification  for  multilayer  networks.  The 
MADALINE  (many  ADALINE)  multi  layer  network  is  also  discussed.  Chapter  4 
discusses  optics  for  neural  networks  with  special  emphasis  on  Polarization  and  Spatial 

Light  Modulators  (SLM’s).  Chapter  5  describes  development  and  implementation  of  an 
optoelectronic  neural  network  and  discusses  input  dependent  thresholding.  Driver 
electronics  for  the  electro  optical  components  are  discussed.  Software  pseudocode  is 
also  presented.  Chapter  6  describes  the  MADALINE  RULE  2  (MR  2)  training  algorithm 
and  the  justification  for  its  implementation.  A  training  set  is  presented  and  results  are 
shown.  Chapter  7  concludes  the  thesis  and  presents  recommendations  for  future  work. 


CHAPTER  2 
Foundation 


The  human  brain  outperforms  the  fastest  digital  computers  in  areas  of  association, 
categorization,  generalization,  and  feature  extraction.  It  consists  of  about  1010  -1011 

neurons,  each  making  about  lO'MO4  synaptic  interconnections  with  neighboring 

* 

neurons  for  a  total  of  1013-1015  interconnections.  The  brain  operates  at  approximately 
100  Hz.  This  means  that  the  brain  functions  at  about  1016  interconnections  per  second 
[2].  A  cross  sectional  view  of  the  cerebral  cortex  (main  computing  apparatus  of  the 
brain)  reveals  three  to  six  layers  of  neurons  similarly  arranged.  This  corresponds 
roughly  to  about  500  neural  network  modules  [3]. 

2.1  Biology 

ANN's  are  biologically  inspired  and  are  made  up  of  elements  analogous  to  that  of  the 
biological  neuron  [3,5].  A  biological  neuron  consists  of  axons,  dendrites  and  synapses 
(Figure  2.1  )[2],  The  soma ,  or  nerve  cell,  is  the  large  central  round  body  of  the  neuron. 
The  axon,  a  thin  extended  fiber  attached  to  the  soma,  produces  and  conducts  electrical 
pulses.  Dendrites  extend  from  the  cell  body  to  other  neurons  where  they  passively 
receive  electrical  signals  at  a  connection  point  called  a  synapse.  At  the  synapse’s 
receiving  end,  inputs  are  conducted  to  the  cell  body  where  they  are  summed.  Inputs 
can  either  excite  or  inhibit  the  cell.  When  the  cumulative  excitation  in  the  cell  body 
exceeds  a  threshold,  the  cell  fires,  propagating  a  signal  down  the  axon  to  other  cells[5]. 
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Figure  2.1.  Biological  Neuron 

2.2  Artificial  Neuron 

An  artificial  neuron  (Figure  2.2)  is  a  simple  processing  element  which  emulates  its 
biological  counterpart.  Processing  elements  are  organized  in  a  way  that  may  or  may  not 
be  related  to  the  anatomy  of  the  brain,  but  surprisingly  exhibit  a  number  of  the  brain’s 
characteristics  [3,5].  Processing  elements  sum  weighted  inputs.  The  weights  can  be 
either  inhibitory  (negative)  or  excitatory  (positive)  with  each  weight  corresponding  to  the 
"strength"  of  a  single  biological  synaptic  connection.  To  better  understand  the  artificial 
neuron  model  one  must  first  be  introduced  to  a  general  framework  for  ANN's. 


FIGURE  2J2.  ARTIFICIAL  NEURON 
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2.3  Framework 

The  general  framework  of  ANN  models  is  described  by  Rumelhart,  Hinton,  and 
Williams  [1 1].  There  are  eight  major  aspects  common  to  most  ANN  models.  These  are: 

1 .  A  set  of  processing  neurons 

2.  A  state  of  activation 

3.  An  output  function  for  each  neuron 

4.  A  pattern  of  connectivity  tor  each  neuron 

5.  A  propagation  rule  for  propagating  patterns  of  activities  through  the  network  of 
connectivities 

6.  An  activation  rule  for  combining  the  inputs  impinging  on  an  neuron  with  the 
current  state  of  that  neuron  to  produce  a  new  level  of  activation  for  the  neuron 

7.  A  learning  rule  whereby  patterns  of  connectivity  are  modified  by  experience 

« 

8.  An  environment  within  which  the  system  must  operate 

2.3.1  Set  of  processing  neurons 

ANN  models  must  specify  a  set  of  processing  neurons  and  what  they  represent. 
These  neurons  may  represent  conceptual  objects  such  as  words,  features,  or  letters. 
They  may  also  represent  abstract  elements  which  define  meaningful  patterns.  There  are 
three  kinds  of  neurons,  input,  hidden,  and  output  (Figure  2.3).  Input  neurons  receive 
inputs  from  sources  external  to  the  system  under  study.  Output  neurons  send  signals 
out  of  the  system.  The  inputs  and  outputs  of  hidden  neurons  are  not  directly  accessible. 
Suppose  there  are  N  neurons.  These  neurons  can  be  numbered  arbitrarily  since  each 

L _ _ 
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neuron  is  essentially  a  simple  processing  element  carrying  out  a  simple  process.  This 
process  receives  inputs,  computes  an  output  value,  and  sends  this  value  to  other 
neurons. 

2.3.2  State  of  activation 

A  representation  of  the  state  of  the  system  is  needed  at  time  k.  This  is  normally 
specified  by  a  vector  of  N  real  numbers,  X^,  representing  the  pattern  of  activation  over 
the  set  of  processing  neurons.  Each  element  of  Xj<  stands  for  the  activation  of  the 
neuron  at  time  k  (Figure  2.2). 

The  pattern  of  activation  over  the  set  of  neurons  captures  what  the  system  is 
representing  at  any  time.  Activation  values  may  be  continuous  or  discrete.  If  continuous, 
values  may  be  bounded  or  unbounded.  If  discrete,  values  may  take  binary  or  a  small  set 
of  values.  Binary  values  are  often  denoted  by  {-1  ,+1),  or  in  some  cases  {0,+1}.  Non 
binary  discrete  values,  for  example,  could  be  restricted  to  the  set  {-1,0+1}. 

2.3.3  Output  function 

Neurons  interact  by  transmitting  signals  to  neighboring  neurons.  The  strength  of  their 
signal  and  the  degree  by  which  neighboring  neurons  are  affected,  is  determined  by  the 
neuron’s  degree  of  activation.  Associated  with  each  neuron  (Figure  2.2),  there  is  an 
output  function  which  maps  the  current  state  of  activation  to  an  output  signal,  OUT*  F 
(NET),  where  NET  =  X^W.  This  function  is  usually  a  threshold  function,  or  a  sigmoidal 
function. 
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2.3.4  Pattern  of  connectivity 

Since  neurons  are  connected  to  one  another,  it  Is  the  pattern  of  conn  ry  that 
constitutes  what  the  system  knows  and  determines  how  it  will  respond  to  arbitrary 
inputs.  Normally,  we  assume  that  each  neuron  additively  contributes  to  the  input  of  the 
neurons  to  which  it  is  connected.  Each  neuron's  output  is  s imply  a  weighted  sum  jf  the 
separate  inputs  from  each  of  the  connected  neurons.  The  pattern  of  connectivity  is 
normally  represented  by  a  weight  matrix,  W,  with  individual  connection  weights  denoted 
by  Wjj  (Figure  2.3).  Weight  Wjj  is  positive  if  the  input  associated  with  it  excites  the 
neuron,  negative  if  it  inhibits  the  neuron,  and  zero  if  it  has  no  direct  connection  to  the 
neuron. 

« 

Input  Neurons  Hidden  Neurons  Output  Neuron 


Weight  connectivity  matrix-W 


Figure  2.3.  Neurons  connected  by  weight  matrices  W. 

The  pattern  of  connectivity  is  very  important  because  it  determines  what  each 
neuron  represents.  How  much  information  can  be  stored  and  how  much  serial 
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processing  the  network  must  perform  is  the  fan-in  and  fan-out  of  each  neuron[1 1].  Fan- 
in  is  the  number  of  elements  that  either  excite  or  inhibit  a  given  neuron.  Fan-out  refers 
to  the  number  of  neurons  affected  by  a  single  neuron. 

2.3.5  Propagation  rule 

The  rule  of  propagation  takes  the  output  vector  representing  the  output  values  of  the 
neurons  and  combines  it  using  connectivity  weight  matrix  W  to  produce  a  net  input, 

NET,  for  each  type  of  input  (inhibitory  or  excitatory)  into  the  next  neuron.  This  is 
expressed  as  NET  =  WXk. 

2.3.6  Activation  rule 

« 

This  rule  takes  the  combined  net  inputs  of  each  type  impinging  on  a  particular 
neuron  and  combines  them  with  the  current  state  of  the  neuron  to  produce  a  new  state 
of  activation,  in  other  words,  function  F  takes  \  and  vectors  NET  for  each  different  type 
of  connection  and  produces  a  new  state  of  activation,  ie  Xk+1  =  F(NET),  where  F  itself  is 
the  activation  rule.  Activation  functions  may  be  a  simple  linear  function, 

OUT  =  K(NET)  where  K  is  a  constant;  a  threshold  function  O  =  1  if  NET  >  T  and 
OUT  =  0  otherwise,  where  T  is  a  constant  threshold  value;  or  a  sigmoid  (squashing) 
function  OUT=  1/(1  +  e  4IET).  These  are  by  no  means  the  only  activation  functions,  but 
are  used  in  many  models  described  in  the  literature^  through  12]. 
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2.3.7  Learning  rule 

Learning  rules  involve  modifying  the  patterns  of  connectivity  as  a  function  of 
experience.  In  principle  this  can  involve  three  kinds  of  modifications: 

1 .  Development  of  new  connections 

2.  Loss  of  existing  connections 

3.  Modifications  of  strengths  of  existing  connections. 

All  rules  of  this  type  are  variants  of  the  Hebbian  learning  rule  [5,1 1].  If  a  neuron  i 
receives  an  input  from  another  neuron  j,  and  if  both  are  highly  active,  weight  Wjj  from 
neuron  i  to  neuron )  should  be  strengthened.  This  is  shown  in  equation  (2.1). 

wjj(k+1)  =  wjj(k)  +  a  OUTpUTj  (2.1) 
where  wjj(k)  *  the  value  of  connection  weight  from  neuron  i  to  neuron  j  prior  to 
adjustment,  Wjj(k+1)  =  the  value  of  connection  weight  from  neuron  i  to  neuron  j  after 
adjustment,  a  =  the  learning-rate  coefficient,  OUTj  =  output  of  neuron  i  and  input  to 
neuron  j,  and  OUTj  =  output  of  neuron  j. 


CHAPTER  3 
ADALINE 

The  adaptive  linear  combiner,  or  non  recursive  adaptive  filter,  was  developed  by 

Widrow  and  Steams  [66]  and  is  the  basic  building  element  used  in  many  neural 

networks  (Figure  3.1).  The  ADALINE  functions  as  an  adaptive  threshold  logic  element . 

In  digital  implementation,  an  input  vector  X«=  [x0  x1k  x2kx3fc....xnk]  is  applied  at  time  k 

and  is  weighted  by  a  set  of  coefficients  which  form  a  vector  w1k  w2k  w3lc... 

wnl}.  The  ADALINE  produces  a  scalar  analog  output  or  inner  product  yk-  Xk  Wk  = 

WkT  Xk.  Weight  vector  element  wok  is  usually  selected  as  the  “bias"  weight  and  is 

normally  connected  to  a  constant  input  x0~  +1 .  A  binary  output,  is  then  computed 

by  the  ADALINE  based  on  scalar  output  yk  Decisions  are  made  by  a  two  level 

« 

quantizer,  where  qk=  SGN  (yk)  for  the  case  of  symmetric  (+1  ,-1)  inputs.  For  binary  (0  or 
1)  inputs,  qk=  1  if  yk=>  0,  otherwise  qk  =0. 

The  desired  response  signal  is  used  to  train  the  neuron.  During  training,  input 
patterns  and  desired  responses  are  presented  to  the  ADALINE.  An  adaptation  algorithm 
adjusts  the  weights  by  minimizing  error  between  the  output  and  the  desired  output.  For 
practicality,  the  adaptation  process  is  oriented  toward  minimizing  the  mean-square 
value  (MSE)  or  average  power  of  the  signal.  The  least  squares  adaptation  algorithm  or 
LMS  [16,66]  (Appendix  1)  or  the  Widrow-Hoff  Delta  Rule  minimizes  the  sum  of  the 
squares  of  the  errors  over  the  training  set  The  desired  response  and  the  components  of 
Xk  could  be  analog  or  binary.  Once  the  weights  are  adjusted  and  the  neuron  is  trained, 
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responses  can  be  tested  by  applying  a  set  of  input  patterns  which  contains  elements  not 
included  In  the  training  set  If  the  neuron  responds  correctly  (with  high  probability),  then 
it  is  said  that  generalization  has  taken  place. 


Figure  3.1.  Adaptive  Linear  Combiner  or  ADALINE 


3.1  Linear  separability 

Linear  separability  is  useful  for  explaining  how  ADALINE  neurons  make  decisions.  It 
also  illustrates  the  limitations  of  single  layer  networks.  Consider  a  neuron  with  n  binary 
inputs  and  one  binary  output  With  n  inputs,  there  are  2n  possible  Input  patterns  and  2 

pn 

boolean  logic  functions  connecting  n  inputs  to  a  single  output  A  single  neuron  with  n 
binary  inputs  is  capable  of  realizing  only  a  subset  of  all  possible  boolean  logic  functions. 
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Figure  3.2  shows  all  possible  binary  inputs  for  a  two-input  neuron  in  pattern  vector 
space.  The  coordinate  axes  are  the  components  of  the  input  pattern  vector.  Depending 
on  the  weight  values,  a  critical  thresholding  condition  occurs  with  the  analog  response , 


y  =  0,  that  is: 


which  yields 


y=xtw1  +  Wq  =  0  (3.1) 


X2=-(W0/W2)-(W1/ 


(3.2) 


The  three  weights  determine  the  slope  (-w0  /  w2) ,  the  intercept  (-vv  /  w^) ,  and  the  side  I 

of  the  separating  line  that  corresponds  to  a  positive  output.The  opposite  side  of  the  line 

t 

r 

then  corresponds  to  a  negative  output. 

i 


Figure  3.2 

The  binary  inputs  are  tabulated  in  table  3.1  This  is  an  example  of  a  linearly  separable 
function.  For  a  nonlineariy  separable  function,  no  single  line  exists  that  can  achieve  the 
separation  of  the  input  patterns.  An  example  is  of  a  nonlineariy  separable  function  is  the 
EX-OR  function  (table  3.2). 
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TABLE  3.1 

TABLE  3.2 

xl  x2 

_ V 

xl 

x2 

y 

+1  +1 

■  ■ 

+1 

+1 

+1 

+1 

+1  -1 

+1 

+1 

►  -1 

-1 

-1  -1 

+1 

-1 

-1 

+1 

-1  +1 

-1 

-1 

+1 

-1 

Table  3.3  summarizes  the,number  of  linearly  separable  functions  for  n  inputs.  It  can 
be  seen  that  the  percentage  of  linearly  separable  functions  decreases  exponentially  with 
increasing  n  and  thereby  limits  the  classification  capability  of  single  neurons  (single 
layer  networks). 


n 

2  2n 

1 

4 

4 

2 

16 

14 

3 

256 

104 

4 

65,536 

1,882 

5 

4.3  x  109 

94,572 

6 

1.8X1019 

5,028,134 

Table  3.3.Number  of  Linearly  Separable  Functions 
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3 J2  Networks 

A  single  neuron  can  perform  simple  pattern  detection  functions.  However,  the  power 
of  neural  computation  comes  from  connecting  neurons  into  networks  (single  or  multi 
layer)  [4,5,7,9,10,1 1] .  In  single  and  multilayer  networks,  Wk  becomes  a  weight  matrix 
rather  than  a  weight  vector.  One  can  therefore  connect  many  ADALINE’s  together  to 
form  MADALINE  networks  which  have  greater  processing  capability.  Lippman  explains 
that  multilayer  networks  perform  more  general  classifications  by  separating  points 
contained  in  convex  open  or  closed  regions  [4].  A  convex  region  is  described  as  having 
any  two  points  which  can  be  joined  by  a  straight  line  that  does  not  leave  the  region.  A 
closed  region  is  one  in  which  all  points  are  contained  within  a  boundary  (ie,  circle).  An 
open  region  has  some  points  outside  any  defined  boundary  (ie,  region  between  two 
parallel  lines).  To  better  understand  convexity,  consider  a  two  layer  (2-2-1)  network  with 
output  neuron  threshold  set  at  0.75.  The  output  neuron  receives  signals  with 
connection  strengths  of  0.5  from  layer  2  neurons.  It  can  be  seen  that  the  output  neuron 
performs  a  logical  "AND*  function  since  both  neurons  in  layer  2  need  to  be  *1"  for  the 
output  neuron  to  exceed  its  threshold.  In  Figure  3.3  it  is  assumed  that  each  neuron  in 
layer  1  subdivides  the  x-y  plane,  with  one  neuron  producing  an  output  of  "1"  for  inputs 
below  the  upper  line  and  the  other  producing  an  output  of  *1"  for  inputs  above  the  lower 
line.  A  v-shaped  region  then  results,  in  which  the  output  neuron  is  one  onfy  over  this  v- 
shaped  region.  If  enough  neurons  are  included  in  the  input  layer,  a  convex  polygon  of 
any  desired  shape  can  be  formed[5].  However,  points  not  comprising  a  convex  region 
cannot  be  separated  from  all  other  points  in  the  plane  by  a  two  layer  network.  The 
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output  of  a  neuron  can  also  produce  functions  other  than  the  logical  aANDa  by  suitably 
choosing  weights  and  thresholds.  For  continuous  inputs,  the  network  can  subdivide  the 
plane  into  continuous  regions  rattier  than  separate  discrete  regions. 

Layer  2  neuron 
is  only  “1"  In 


FIGURE  3.3.  CONVEX  DECISION  REGION  PRODUCED  BY  A  2-2-1  NETWORK 
If  another  2-2-1  network  is  used  in  conjunction  with  the  first  network,  two  separate 
convex  decision  regions  are  created.  These  two  regions  can  then  be  joined  by  a  third 
layer  neuron  as  shown  in  Figure  3.4.  Classification  capacity  is  now  limited  only  by  the 
number  of  artificial  neurons  and  by  the  weights.  There  are  no  convexity  constraints[4,5j. 

It  has  been  shown  that  a  3  layer  network  is  capable  of  realizing  any  classification 
function  [16].  A  3  layer  network  was  therefore  attempted  classifying  patterns.  The  first 
two  layers  were  implemented  optically  since  there  were  only  two  optical  processors,  and 
the  output  layer  was  done  electronically  in  the  computer  memory. 


CHAPTER  4 

OPTICS  FOR  NEURAL  NETWORKS 

Optics  offer  many  advantages  which  can  be  exploited  for  developing  ANN’S.  One 

* 

advantage  is  higher  bandwidth  over  that  of  electronics  because  unlike  electronics,  there 

is  no  capacitance  present  in  optical  connections  allowing  for  faster  transmission  of 

signals.  Another  advantage  results  from  photons  not  interacting  with  one  another  unlike 

electrons  (because  of  charge).  Consequently,  light  beams  may  pass  through  one 

another  without  distorting  the  information  carried  [63]. 

However,  a  main  disadvantage  of  optics  is  that  there  exist  few  devices  which 

perform  all  optical  signal  processing  functions,  and  these  are  still  in  development  and 

« 

not  accessible.  Until  these  devices  are  further  developed  (SEED)[63],  electro  optical 
devices  provide  a  means  by  which  signals  can  be  applied  electronically,  converted  to  an 
optical  equivalent  for  optical  processing,  and  then  converted  back  to  an  electronic  signal 
which  can  be  read  or  stored  by  a  digital  computer. 

4.1  Polarization 

Polarization  is  significantly  important  in  optical  computing  because  it  enables  some 
operations  to  be  performed  with  very  little  energy  loss,  such  as  switching  states  or 
signal  steering  [63].  Polarization  is  also  essential  for  using  a  Spatial  Light  Modulator 
(SLM)  discussed  in  Section  4.3. 
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Electromagnetic  waves  such  as  light,  have  electrons  oscillating  transversely  to  the 
direction  of  wave  propagation,  z  for  example,  In  all  directions  of  the  x-y  plane  [63]. 
There  are  three  forms  of  polarization:  linear,  circular,  and  elliptical  [62,63]. 

A  polarizer  is  an  optical  device  whose  input  is  natural  or  randomly  polarized 
light  and  its  output  is  some  form  of  polarized  light.  Polarizers  are  categorized  as 
linear, circular  or  elliptical  and  are  all  based  on  one  of  four  fundamental  physical 
mechanisms:  dichroism  or  selective  absorption,  reflection, scattering,  and 
birefringence  or  double  refraction.  When  used  as  analyzers,  polarizers  allows 
light  polarized  in  only  one  direction  to  pass.  Polaroid  HN  38  material  is  dichroic 
material  which  selectively  absorbs  one  of  the  two  orthogonal  polarized 
components  of  an  incident  beam  and  is  widely  available  [62]. 

4.2  Spatial  Light  Modulators 

A  spatial  light  modulator  produces  an  output  of  light  that  is  modulated  over  the 
plane  perpendicular  to  the  direction  of  propagation.  The  modulation  is  generally  a 
change  in  the  angle  of  polarization,  which  in  turn  can  be  translated  into  an  amplitude 
modulation  based  on  the  angle  of  rotation  and  an  output  polarizer. 

Modulation  of  the  output  beam  is  done  by  the  beam’s  interaction  with  the  active 
medium  which  is  controlled  by  a  field  applied  to  it  [33].  SLM’s  are  constructed  from 
either  liquid  crystal  [35,39],  ferro-electric  liquid  crystal  [34],  magneto-optic  materials 
[37], or  micro-channel  plate  [36].  Other  types  use  deformable  mirrors  [40,41,42]  and 
operate  on  the  phase  of  the  light  and  not  its  amplitude.  Spatial  Light  Modulators  can  be 
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addressed  optically  or  electronically.  In  electronic  addressing ,  the  optically  active  areas 
are  defined  as  pixels  and  arranged  as  an  n-by-n  matrix.  A  computer  stored  Image  can 
then  be  written  to  the  SLM  via  interface  electronics  by  applying  a  voltage  or  current  to 
the  respective  X-Y  electrical  connections. 

SLM’s  perform  optical  signal  steering,  multiplication,  and  memory  storage  as 
illustrated  in  Figure  4.1.  For  multiplication,  the  SLM  realizes  a  vector-matrix 
multiplication  of  vector  L  containing  elements  {L1,L2,L3,L4}  with  matrix  A,  yielding 
vector  D  containing  elements  {D1,D2,D3,D4},  where  D=LTA.  If  matrix  coefficients  are 
either  *0'  or  “1 "  the  SLM  realizes  an  interconnection  array  routing  signals  from  L  to  D. 

LED’s 


Figure  4.1.  Spatial  Light  Modulator 


The  SLM's  used  In  this  thesis  study  were  custom  manufactured  and  constructed  with 
liquid  crystal  material.  Each  SLM  has  a  16X16  weight  matrix  and  is  driven  in  the  direct 
drive  method  with  a  IBM  compatible  computer  controlling  the  pixel  transmission  level. 
Since  the  transmissive  mode  is  used,  input  and  an  output  polarizers  are  needed.  The 
polarizing  material  is  Polaroid  HN  38. 

4.3  Emitters 

There  are  many  forms  of  light  emission  but  not  all  can  be  used  in  optical 
implementation  of  neural  networks.  A  useful  device  is  the  light  emitting 
diode  or  LEO.  Semiconductor  lasers  are  of  special  interest  for  optical  neural 
networks  for  the  following  reasons[63]: 

•  Devices  are  small,  approximately  300  X 10  X  50  microns 

•  Voltages,  currents,  and  power  are  low,  typically  2V  and  15-100  mA 

•  Efficiency  is  high,  on  the  order  of  20% 

•  High  modulation  rates,  on  the  order  of  10  GHz 

•  Devices  interface  easily  with  integrated  optics  and  integrated  electronics 

•  Low  cost 

Arrays  of  laser  diode  and  light  emitting  diode  sources  provide  an  electronically 
addressable  SLM  with  good  intensity  and  equal  spot  illumination.  Light  is 
emitted  only  from  diodes  which  have  a  voitage  applied.  Image  information 
stored  in  a  computer  can  be  transferred  to  an  optical  network  with  such  an 
array  (63]. 
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4.4  Detectors 

Optical  detectors  convert  optical  signals  into  electronic  signals.  Detectors 
can  be  classified  into  four  categories  [61]. 

1 .  External  photoelectric  effect  detectors.  These  include  vacuum  photodiodes 
and  photomultipliers. 

2.  Interna)  photoelectric  effect  detectors.  This  covers  a  wide  range  of 
semiconductor  devices  in  which  photons  are  absorbed  to  produce  charge 
carriers. 

3.  Thermal  detectors.  These  use  the  direct  open  effect  of  absorbed 
radiation  and  include  bolometers  (resistance  thermometers)  and 
thermocouples. 

4.  Detectors  in  which  a  chemical  change  is  initiated  by  optical  radiation,  such 
as  in  the  human  eye. 

Semiconductor  or  junction  photodiodes  are  primarily  used  in  neural  network 
optical  detection  [63].  A  characteristic  of  these  detectors  is  that  if  no  bias  is 
applied,  they  generate  a  photovoltaic  voltage  proportional  to  the  amount  of 
incident  light. 


CHAPTER  5 

OPTICAL  NEURAL  NETWORK 


Optical  neural  networks  based  on  Vector-Matrix  multipliers  have  been  configured  as 
associative  memories  [18, 20, 22, 23, 28, 29, 51],  and  pattern  classifiers  [22, 24, 25, 26, 
27,30, 31].  The  function  of  the  weight  mask  is  to  produce  a  dynamically  variable 
transmittance  proportional  to  the  desired  connection  weight  The  use  of  cylindrical 
lenses  for  signal  steering  is  not  practical  if  this  type  of  network  is  to  be 
implemented  at  the  micro-optical  and  micro-electronics  level.  Therefore  crossbar  card- 
readers  which  use  waveguides  for  light  distribution  and  collection  were  used. 

The  network  consisted  of  3  layers.  The  first  two  layers  were  realized  qpticaily 
with  SLM’s  used  as  vector  matrix  multipliers.  Each  optical  layer  has  a  16X1  input 
where  one  of  the  inputs  served  as  a  bias  input, and  a  16X1  output  vector  where 
one  of  the  output  elements  is  needed  for  input  dependent  thresholding  (see 
Section  5.2).  The  third  layer  has  a  16X1  input  with  one  input  serving  as  a  bias 
input  and  a  3X1  output  Input  patterns  are  presented  as  5X3  2-D  data  mapped  as 
a  one  dimensional  15X1  vector. 

5.1  Unipolar  vs  Bipolar  SLM  arrangement 

Most  previously  mentioned  optoelectronic  networks  utilize  one  pair  of  SLM’s  per 
layer.  One  SLM  stores  positive  weights  and  the  other  stores  negative  weights  since  one 
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cannot  have  "negative*  light .  Output  detectors  are  then  coupled  to  differential 
amplifiers  which  provide  a  weighted  sum  for  each  neuron.  This  technique  increases 
network  complexity  and  cost  In  unipolar  mode,  one  SLM  is  used  per  layer  to  realize 
both  positive  and  negative  weights.  This  was  accomplished  by  means  of  a  positive 
weight  offset  given  to  each  connection. 

5.2  Input  Dependent  Thresholding 

For  each  output  matrix  W  new  weights  are  W^*  =  p,  where  p  represents  a 
positive  offset.  Because  yk=Xwik'  Xk  =  X(Wk+p)Xk  =  £wlkXk  +  £  p  xk ,  the 
threshold  function  is  dependent  on  the  varying  input.  Positive  offset  p  is  determined 
from  the  spatial  light  modulator  gray  scale  range.  For  example,  if  the  gray  scale  range  is 
120  levels  per  pixel,  the  minimum  negative  weight  value  could  be  set  at  -59  and  the 
maximum  positive  value  could  be  set  at  60  (because  zero  counts  as  a  weight).  Positive 
offset  p  is  then  required  to  have  the  value  of  60  in  order  for  the  new  weights  to  span  a 
range  of  1  to  120. 

Hardware  realization  of  input  dependent  thresholding  is  realized  by  having  a 
threshold  column  in  which  each  of  the  column’s  pixels  are  set  to  half  of  maximum 
contrast  (Figure  5.1).  The  value  of  the  photodiode  detector  which  receives  the 
weighted  sum  of  the  inputs  through  the  threshold  column  now  becomes  the  two  level 
quantizer  decision  value. 
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Figure  5.1.  Input  Dependent  Thresholding  Configuration 


5.3  Driver  and  interface  electronics 

The  optical  ANN  system  is  functionally  made  up  of  seven  sections.  These  are: 

•  Programmable  liquid  crystal  display  (LCD)  drivers 

•  Spatial  light  modulators 

•  Light  emitting  diode  or  LED  linear  array  stages 

•  Linear  array  photodiode  receivers 

•  Analog  signal  conditioning 

•  Digital  and  analog  interface 

•  Training  software 


A  block  diagram  of  the  optoelectronic  neural  network  (Figure  5.2)  illustrates  the 
signal  flow  of  the  system.  The  system’s  software  controls  an  ADC  board  and  a  digital 
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Figure  5.2.  System  overview 

input/  output  (I/O)  board.  The  ADC  board  houses  a  multiplexed  48  channel  12  bit 
successive  approximation  analog  to  digital  converter.  The  first  32  channels  receive 
amplified  photodiode  signals  which  represent  output  vector  analog  elements.  The 
digital  I/O  board  consists  of  96  channels  of  I/O  capacity.  This  controls  the  two  linear 
array  LED  boards  which  provide  an  input  vector  signal  to  the  SLM’s,  the  programmable 
SLM  driver  boards,  and  reads  the  outputs  of  the  nonlinear  activation  function 
(comparators)  boards.  Except  for  the  ADC  and  digital  I/O  boards,  the  system’s 
hardware  is  housed  in  a  19  inch  electronic  cabinet 


5.3.1  Programmable  drivers 

Programmable  drivers  control  the  gray  scale  value  of  each  pixel  in  the  spatial  light 
modulator.  Each  spatial  light  modulator  contains  256  pixels.  Since  two  modulators 
were  used,  a  total  of  512  independent  programmable  drivers  were  needed.  The 
programmable  drivers  were  designed  with  the  goal  of  minimizing  circuit  complexity. 

This  was  realized  with  the  design  of  a  driver  "ceir  shown  in  Figure  5.3.  The  cell  was 
then  replicated  16  times  to  fit  on  a  12"  x  12’  printed  circuit  board  (Appendix  2).  The 
programmable  driver  printer  circuit  board  provided  32  pixel  driver  signals.  Sixteen  driver 
printed  circuit  boards  were  manufactured  for  driving  512  individual  pixels. 


Figure  5.3.  Driver  Electronics  (512  each) 

Pixel  transmission  programming  is  accomplished  in  two  steps.  A  total  of  512 
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independent  0-5  volts  DC  LCD  waveforms  are  generated  through  1 6  cascadable  LCD  1 

generator  chips  Hughes  model  H0438A.  The  H0438A  is  a  CMOS/LSI  circuit  which  ' 

drives  field  effect  or  dynamic  scattering  LCD  displays.  . 
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Figure  5.4.  Hughes  H0438A  LCD  Driver 


Each  H0438A  (Figure  5.4)  generates  up  to  32  pixel  driving  waveforms  with  correct 
phase  and  refresh  rate  for  each  LCD  pixel.  The  H0438A  contains  a  32  bit  static  shift 
register,  32  latches,  an  LCD  phase  generator,  and  32  segment  (pixel)  drivers  (Figure 
5.5).  Information  is  written  to  the  32  bit  shift  register  through  a  three  wire  serial  port. 
The  three  wire  serial  port  consist  of  Data  In,  Load,  and  Clock.  Data  is  entered  into  the 
32  bit  static  shift  register  only  when  Load  is  high.  Data  is  entered  while  Load  is  high 
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through  the  Data  In  pin  on  the  foiling  edge  of  Dock.  Data  Is  loaded  starting  with  the 
value  for  segment  32  down  to  segment  1 .  A  logic  1  on  Data  In  causes  a  segment  to  be 
visible.The  H0438A  can  be  cascaded  by  connecting  the  Data  Out  of  the  first  device  to 
the  Data  In  pin  of  the  next  device.The  LCD  clock  signal  input  pins  share  a  common 
driving  signal. 


DATA  IN 
CLOCK 

LOAD 


LCD 

CLOCK 


I 


SEG  SEG  SEG 
1  2  3 


LCD  AC 
generator! 


SEG  SEG  SEG 
30  31  32 


> 


DATA  OUT 


Backplane 

Output 


Figure  5.5.  H0438A  block  diagram 

Clock  and  Load  signals  are  also  shared  by  all  cascaded  devices.  As  data  is  docked  into 
the  first  device  through  Data  In,  previously  stored  information  from  the  static  shift 
register  is  clocked  out  bit  by  bit  and  into  the  next  device. 

If  a  DC  voltage  were  applied  to  a  pixel,  it  would  destroy  the  pixel.The  H0438A 
produces  square  waves  with  5  volt  amplitude.  The  frequency  of  the  output  square 
waves  is  equal  to  the  LCD  dock  input  frequency.  A  backplane  output  square  wave  is 
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also  generated  and  serves  as  a  phase  reference.  Figure  5.6  illustrates  the  purpose  for 
having  a  backplane  signal.  If  both  backplane  and  segment  waveforms  are  in  phase,  the 
pixel  is  essentially  "off*.  If  the  segment  signal  is  90  degrees  out  of  phase  with  the 
backplane  signal,  the  pixel  is  'on*. 


r  JTJTrLTL 

~~-r  JinrLTL 


Backplane 


BACKPLANE 


jinnrL 

jinnrL 


Figure  5.6.  Segment  (pixel)  driving. 


A  LM555  timer  chip  is  used  to  generate  a  100  Hz  LCD  clock.  Figure  5.7  describes 
the  LCD  clock  generator.  This  circuit  cannot  realize  a  50%  duty  cycle  square  wave,  but 
a  close  approximation  was  adequate.  Values  for  the  circuit  elements  are:  Ra=  1M, 

Rb*  220K,  and  Ct  =  0.01  uF.  These  yield  approximately  a  100  Hz  LCD  clock. 

The  0-5  volts  DC  waveforms  are  attenuated  by  a  voltage  divider  resistor  circuit.  The 
non-inverting  input  receives  a  square  wave  with  a  17  mV  amplitude.  Dallas 
Semiconductor  DS1267  (Figure  5.8)  dual  programmable  potentiometers  are  used  in  the 


feedback  of  the  non  inverting  gain  stages  and  provide  computer  controlled  gain  for  each 
waveform. 


_rui_ 

100  Hz 
CtaO.01  up 

Ra  ■  1M 

Rb  >  220 K 


TIb  0.693  (Ra+Rb)Ct 
T2  a  0.683  (Rb)  Ct 
T  a  T1  ♦  T2 
Fb1/T 

Figure  5.8.  LCD  clock  generator 


The  DS1267  contains  two  potentiometers,  each  of  which  has  its  wiper  set  by  a  value 
contained  in  an  8  bit  register  (Figure  5.9).  Each  potentiometer  consists  of  256  resistors 
of  equal  value  with  tap  points  between  each  resistor  and  the  low  end.  An  8  bit  wiper 
register  controls  a  256-to-1  multiplexer  (mux)  that  selects  which  tap  point  is  connected 
to  the  wiper  output. 
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DS1267 


Figure  5.8.  DS1267  Dual  Potentiometer 
Information  is  written  and  read  from  each  wiper  stack  register  via  a  17  bit  I/O 
shift  register.  The  I/O  shift  register  is  serially  loaded  by  a  3  wire  serial  port.  The  three 
wire  serial  port  consist  of  RST\,  DQ,  and  clock.  Data  is  entered  into  the  17  bit  shift 
register  only  when  RSTV  is  high.  Data  is  entered  while  RSTV  is  high  through  the  DQ  pin. 


Figure  5.9.  DS1267  Block  diagram 
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The  potentiometers  always  maintain  their  previous  value  until  RST\  is  set  to  a  low 
level,  which  terminates  data  transfer.  Valid  data  is  clocked  in  on  the  low  to  high 
transition  of  the  CLK  input  Data  is  entered  with  the  Stack  Select  bit  starting  the  data 
stream,  followed  by  the  LSB  of  wiper  1 ,  MSB  of  wiper  1 ,  LSB  of  wiper  0  ,  and  terminated 
by  the  MSB  of  wiper  0.  A  total  of  17  bits  constitutes  valid  data.  As  the  data  is  clocked 
into  the  shift  register,  previous  bits  are  shifted  out  bit  by  bit  on  the  cascade  serial  port 
pin,  Cqut*  By  connecting  the  C0UT  to  the  DQ  input  of  another  DS1267,  multiple  devices 
can  be  daisy  chained  together. 

The  non-inverting  gain  stage  has  a  maximum  gain  of  (1  + 1 0K/31)  =  257.  This  gives 
the  programmable  pixel  driver  an  output  amplitude  range  of  34  mV  to  4.36  Volts. 

The  circuit  was  simulated  with  PSPICE  and  its  output  is  shown  in  Figure  5.10.  The 
circuit  configuration  allows  for  the  potentiometer  to  be  useful  for  almost  its  entire  range 
of  values.  However,  the  operational  amplifier  saturates  close  to  its  positive  supply  rail 
value  (5  volts)  for  the  upper  1000  ohms  (9,000-10,000  ohms). 


PROGRAMMABLE  DRIVER 


4000  6000  8000 

POTENTIOMETER  RESISTANCE  •  Ohm* 
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Figure  5.10.  Driver  output  versus  potentiometer  value 
The  programmable  drivers  control  each  LCD  pixel  independently  of  Its  neighbors. 
Since  the  amount  of  polarization  is  dependent  on  the  angle  of  rotation  of  the  LCD 
material  and  the  LCD  material  behaves  nonlinearly  with  respect  to  the  electric  field 
applied  to  it,  a  linear  range  of  approximately  120  gray  levels  (Figure  5.1 1)  is  achieved 
for  each  pixel. 


SLM  Contrast  for  Red  Emmitter 


Figure  5.11.  SLM  pixel  contrast 


5.3.2  Spatial  light  modulator 

The  SLM ’s  are  made  of  specially  etched  glass  and  were  custom  manufactured  by 


U.C.E-  ^  Connecticut  Indium  Tin  Oxide  conductors  were  etched  on  the  glass  and 
provide  electrical  contacts  for  each  pixel.  The  arrangement  of  each  pixel  number  is 
shown  in  Figure  5.12.  The  conductors  are  arranged  in  two  set  of  128  connections 
located  on  either  end  of  the  SLM.  Figure  5.13  shows  the  correspondence  of  pixel 
number  to  weight  matrix  W  element. 


Pixels  0-31  Pixels  1 28-1 59 


Figure  5.12.  Indium  Tin  Oxide  Conductor  Placement 


The  Indium  Tin  Oxide  conductors  were  bonded  with  .025”  pitch  Silicon  Rubber  heat 


seal  connectors  to  two  printed  circuit  boards  for  each  SLM  (Appendix  2).  This  was 
performed  by  ELFORM,  a  company  located  in  Reno.  Driver  board  to  SLM  board 
connections  are  made  with  32  conductor  ribbon  cable  assemblies. 
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Figure  5.13.  Pixel  number  correspondence  to  SLM  weights 
5.3.3  LED  linear  array 

Input  and  hidden  vector  light  signals  are  provided  by  two  linear  arrays  of  16  high 
output  680  nm  peak  light  emitting  diodes  model  A.N.D.  Kilobright  180CRP.  This 
wavelength  was  chosen  because  it  provides  the  maximum  contrast  for  HN  38  polarizing 
material.  The  LED’s  are  driven  with  ULN  2001  LED  driver  IC’s.  Each  ULN2001 1C 
consists  of  7  driver  transistors  similar  to  2N2222  transitors.  Figure  5.14  illustrates  the 
LED  driving  circuit  used. 
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Figure  5.14.  LED  driver  stage 


The  LED’s  were  operated  in  binary  fashion,  i.e.,  they  represented  binary  input  vector 
elements.  No  gray  levels  were  represented.  The  arrays  were  layed  out  on  a  printed 
circuit  board  with  each  LED  spaced  5mm  apart  (Appendix  2).  These  are  connected  to  a 
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digital  interface  board  by  20  conductor  ribbon  cable. 

5.3.4  Photodiode  receivers 

The  output  vector  elements  of  layers  1  and  2  are  realized  electro  optically  with 
matched  photovoltaic  mode  photodiodes,  model  UDT-10  from  United  Detector 
Technologies.  These  diodes  are  arranged  in  a  linear  array  (Appendix  2)  and  provide  a 

m 

weighted  sum  of  each  input.  Two  printed  circuit  boards  house  the  photodiode  receivers, 
one  for  each  layer.  Each  board  is  mounted  next  to  the  output  image  plane  of  the 
SLM  matrix-vector  multiplier.  The  photodiodes  are  spaced  5mm  apart  on  the  printed 
circuit  boards.  Output  signals  are  then  fed  into  corresponding  analog  conditioning 
boards  with  20  conductor  ribbon  cables. 

5.3.5  Analog  signal  conditioning 

The  photodiodes  outputs  cover  a  functional  range  of  40  mV  to  340  mV. 

Each  output  vector  is  fed  into  a  signal  conditioning  board  which  provides  gain. 

The  boards  consists  of  LM324  operational  amplifier  non-inverting  gain  stages 
operating  in  photovoltaic  mode  (Figure  5.15). 
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OUTPUT 


Figure  5.15.  Photodiode  amplifier 

The  non-inverting  gain  stage  circuit  was  simulated  with  PSPiCE.  The  simulation 
result  is  shown  in  Figure  5.16  below. 


Figure  5.16.  PSPICE  simulation  graph  of  photodiode  gain  stage 
There  are  16  amplifiers  on  each  board.  The  output  of  the  amplifiers  are 
connected  to  analog  comparator  boards  by  20  conductor  ribbon  cables,  and 
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ra¬ 
te  a  50  pin  ribbon  cable  connector  from  the  ADC  board. 

The  analog  comparators  (Figure  5.17)  utilize  LM31 1  precision  comparators 
and  realize  the  threshold  activation  function,  i.e.  the  output  is  low  when  Vin  <= 
Vref  and  high  otherwise.  A  PSPICE  simulation  of  comparator  circuit  illustrates  a 
threshold  activation  function  in  Figure  5.18. 

The  output  of  the  comparators  are  digital  and  are  read  by  the  computer 
through  the  digital  interface  board.  The  ADC  is  only  used  to  read  analog  values 
which  are  needed  during  the  training  phase  for  rank  ordering  of  the  optical 
neurons.  The  ADC  is  not  used  for  testing  the  network. 


COMPARATOR  TEST  CIRCUIT 


Figure  5.17.  Nonlinear  activation  circuit 


Figure  5.18.  PSPICE  simulation  of  circuit  in  Figure  5.9 

5.3.6  Analog  and  digital  interface 

The  analog  interface  is  realized  by  a  48  channel  ADC  computer  plug  in  board 
from  Computer  Boards,  Inc.  The  board  was  located  at  base  address  320  hex.  It 
was  used  to  read  32  channels  of  photodiode  data.  The  values  of  the  photodiodes 
represented  the  analog  output  of  each  neuron.  These  values  were  required  for 
training  the  network. 

The  ADC  board  contains  a  12  bit  converter,  a  multiplexer ,  and  48  sample  and 
hold  amplifiers.  The  sample  and  hold  amplifiers  have  a  15  us  acquisition  time  to 
0.01%  of  input  signal  value.  Figure  5.19  provides  connector  pinout  assignment 
for  the  ADC  board  and  the  photodiode  amplifier  outputs. 
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Figure  5.19.  ADC  connector  pinout  assignment 

« 

The  ADC  board  has  4  addresses  which  provide  all  necessary  controls  to  the 
board’s  functions.  The  following  code  reads  the  first  16  channels  of  data  corresponding 
to  output  layer  1  detectors. 

/*•***•  adc 

#define  ADC.BASE  0x0320 

#define  TRIGGER  0 

#define  SET.GAIN  (ADC.BASE  +  3) 

#define  GAIN  3 

#define  ADC.CONVERT  (ADC.BASE +  1)  /*  12  BIT  CONVERSION  V 
#define  SET.CHANNEL  (ADC.BASE  +  2) 
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d&'JiA MB! 

V;'; 


fdefine  READ.LOW  AOC.BASE 

Idefina  READJHIGH  (AOC.BASE  + 1) 

Ktofine  EOC  (AOC.BASE  +  2) 

void  ADC.LAYERI  (void) 
unsigned  Int  channel; 
int  tow_byte,high_byte,adc_counts; 
unsigned  char  check_EOC; 
double  analog; 

/•••••Read  layer  1  channels*"*/ 
for  (channel  ■  0;  channel  <16;  channel++){ 
outportb(SET_CHANNEL, channel); 
outpocttKADC.CONVERT.TRIGGER); 


checR_ECX^inportb(EOC); 
printf(*EOC  -  %u  \n*,checK_EOC); 


while  (checK_EOC  >  127); 

low_byte=inportb(READ_LOW); 

high_byte=lnportb(READ_HIGH); 

adc_counts=((high_byte*  1 6)  +  (low_byte/16));  /TOTAL  ADC  COUNTS  */ 

Hldden1_analog_value[channeQ  -  adc_counts  *  .00122;  /*  12  BIT  CONVERSION  0  -5  VOLT  RANGE  7 


The  digital  interface  is  realized  in  two  parts.  A  Computer  boards  Inc.  CIO-DIO 
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96  digital  I/O  computer  plug  in  board  located  at  base  address  300  hex  provided 

digital  lines  for  programming  H0438A  LCD  waveform  generators  and  OS  1267 

potentiometers,  controlling  individual  LED’s,  and  reading  comparators.  These 

|  signals  were  then  distributed  accordingly  by  a  breakout  board  (Figure  5.19)  which 

:  also  provided  a  100Hz  clock  for  the  H0438A  waveform  generators. 

The  CIO-DIO  96  is  a  96  line  digital  I/O  board  controlled  by  4  82C55  TTL  level 

» 

digital  I/O  chips.  Each  chip  contains  3  data  and  one  control  register  occupying  4 
consecutive  computer  address  locations  (Figure  5.20).  The  CIO-DIO  96  needs 
16  consecutive  computer  I/O  address  locations  to  function  without  conflicting  with 
other  devices.  There  are  two  connectors  on  the  board  denoted  by  PI  and  P2  in 
Figure  5.20.  These  provide  connections  of  external  digital  devices  with  the  CIO- 
DIO  96. 


uiia 


MIM 


SWITCH 


DECODE 


PCflCT/ATMIIPSMO  BUS 


Figure  5.20.  CIO-DIO  96  Block  diagram 
The  CIO-DIO  96  has  simple  control  and  data  register  programming.  Table 
5.1  summarizes  control  and  data  register  functions. 


ADDRESS  READ  FUNCTION 


BASE+0  Port  A  Input  of  8255*1 


BASE+1  |  Port  B  Input 


BASE+2  I  Port  C  Input 


BASE+3  |  None.  Read  back  on  8255 


BASE+4  Port  A I 


BASE+5  I  And  so  on.... 


Table  5.1.  CIO-DIO  96  data  and  control  register  functions 


WRITE  FUNCTION 


Port  A  Output 


Port  B  Output 


Port  C  Output 


ure  8255  *1 


BJJJ 

Of  8255*2 

Port  A 
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The  8255  chips  can  be  configured  in  5  different  modes  of  operation.  Mode  0 
provides  simple  I/O  capability  with  no  handshaking  since  the  software  controls  all 
I/O  events.  The  following  C  code  illustrates  Mode  0  I/O  programming. 


void  MODE_0(vokJ) 


{ 


outportb(BASE+3,128); 
outportb(BASE+7,1 28); 
outportb(BASE+1 1 ,155); 
outportb(BASE+1 5, 1 37); 


r  Mode  0  Output  FOR  U27 
r  Mode  0  Output  FOR  U17 
r  Mode  0  Input  FOR  U87 
I*  Mode  0  Input/Output  FOR  U77 


) 

Connectors  PI  and  P2  are  connected  to  the  interface  board  through  two  sets  of  50 
conductor  ribbon  cables.  The  pinout  assignemt  is  shown  in  table  5.2.  LED’s  are 
numbered  0  to  31  indicating  that  the  first  16  drive  the  input  vector  and  the  remaining 
16  drive  the  hidden  layer  2  input  vector.  Likewise,  comparator  outputs  are  labeled 
PHOTO  0  to  PHOTO  31  where  PHOTO  0-15  connect  the  first  layers  comparator 
output  to  the  computer  and  the  remaining  16  connect  the  comparators  evaluating 
hidden  layer  2’s  output.  LCD  clock  1 ,  LCD  Load  1 ,  Pot  Clock  1 ,  Pot  Data  1 ,  and  Pot 
Load  1 ,  refer  to  weight  layer  1  drivers.  The  others  to  weight  layer  2  drivers. 
Programmable  driver  boards  1  through  16  are  programmed  with  lines  B0-B7,  and 
A0-A7  of  U7. 
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CONNECTOR  PI 
U1  U2 


CONNECTOR  P2 
U7  U8 
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5  A3: 

6  A2: 

7  A1: 

8  AO: 

9  B7: 

10  B6: 

11  B5: 

12  B4: 

13  B3: 

14 
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19  C5; 

20  C4: 
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23  Cl 

24  CO: 
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B4:  BOARDS  DATA 
B3:  BOARD4  DATA 
B2:  BOARD3  DATA 
B1:  BOARD2  DATA 

16  BO:  BOARD1  DATA 
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29 
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32 

33 

34 

35 

36 

37 
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39 

40 

41 

42 

43 

44 

45 
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47 
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A7:PH0T023 


A4:PHOTO20 
A3:PH0T019 
A2:PH0T018 
A1:PHOTO17 
AO:  PH0T016 
B7:  PH0T015 
B6:  PH0T014 
B5:  PH0T013 
B4:  PHOTO  12| 
B3:  PHOTO1 1 
B2:  PHOTOIO 
B1:  PH0T09 
BO:  PH0T08 
C7:  PH0T07 
C6:  PH0T06 
C5:  PH0T05 
C4:  PH0T04 
C3:  PH0T03 
C2:  PHOT02 
Cl:  PHOTOI 
CO:  PHOTOO 
5V 
GND 


Table  5.2.  CIO-DIO  96  pin  assignment 


5.3.7  System  software 

The  system  software  controls  both  analog  and  digital  plug  in  boards,  and 
trains  the  network.  The  system  software  consists  of  12  separate  hierarchically 
shown  in  Figure  5.12.  Menu3.c  provides  the  user  with  training  options  and 
parameters.  Include3.c  contains  all  headers,  definitions,  function  prototypes,  and 
global  variables  used  or  called  by  all  other  files.  Uniforms  contains  functions  for 
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uniformly  distributing  network  weights  during  weight  initialization.  Rank.c 
contains  function  for  array  indexing  and  ranking.  Network3.c  saves  and  recalls 
network  weights  for  training  and  testing  of  patterns.  File.c  provides  file  input  and 
output  capability.  Train.c  provides  a  menu  for  training  the  network.  Test3.c  tests 


Figure  5.12.  Software  hierarchy 


patterns  after  network  has  been  trained.  Share3.c  provides  MR2  training  rule 
functions  shared  by  optical  system  software,  optical  simulation  software,  and  by 
traditional  MR2  algorithm  training.  Drfver3.c  provides  hardware  digital  and 


i 

analog  input  and  output  functions.  Optical3.c ,  Sim3_d.c  and  3layer.c  are  the 
actual  MR2  training  software  for  the  three  systems.  Detailed  system  software  is 
found  in  Appendix  4.  The  files  are  listed  in  'loose*  pseudocode  style  [65]  below. 

PROGRAM :  MENU3.C 

DESCRIPTION:  Provides  simple  menu  interface  for  Optical  ANN.  Allows  user 
to  select  various  modes  of  training. 


get  include3  header  and  definition  file 

x 

SYSTEM  PARAMETERS  function  asks  user  for  choice  of  system  (Optical  or 
Software  only)  and  for  weight  distribution  bounds. 


function  system_parameters() 
/************************“************“*»******************' 
Main  Program 

•••••*****«**ft*«**ft****#«e*##*********#********«***««*«««««**«i 


integer  mainO 
initialize  variables 


Display  the  ANN  Menu 


display  Optical  ANN  Main  Menu 


get  choices  from 


58 


1.  System  parameters 

2.  File  menu 

3.  Train  menu 

4.  Test  menu 

a************************************************************* 


FILE:  Include3.c 

DESCRIPTION:  Contains  headers,  definitions,  and  global  variables  used  in  the 
optical  neural  network  system  simulation.  Refer  to  source  code  since  it  is 
straighforward.. 


FILE:share3.c 

DESCRIPTION:  Contains  functions  shared  by  simulation  and  optical 
hardware  during  training  phase. 


INITIALIZE  WEIGHTS  function  uniformly  distributes  weight  values  in  each 
of  the  weight  layers. 


function  initialize.weightsO 

Weightjayers  1, 2  and  3[i][fl  =  uniform_dist(Min,Max); 

ye************************************************************************ 

INITIALIZE  adapted  function  clears  hidden  neuron  adapted  value. 


function  lnitialize_adapted() 

Initialize  Hidden  vectors  adaptation  control  variable  at  beginning 
of  training 

/•SSttMSttSSSttMSStMSStSSSMSSesSMSSSSiltSSSMMftMMSnSMSSSMSSSMSt 


f 
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BINARIZEJNPUTS  function  reads  msb  and  Isb  input  vector  values  and  assigns 
a  corresponding  +1  or  0.  No  symmetry  Is  used.  This  is  only  used  to 
calculate  hidden  neuron  analog  values. 


function  binarizejnputs(unsigned  char  Isb, unsigned  char  msb) 


^»*****«  *******  *************************************** ********  ************ 

GENERATE  OUTPUT  NEURONS  function  calculates  the  weighted  sum  for  each 
output  neuron. 

function  generate_output_neuronsO 

Perform  matrix-vector  multiplications.  Used  only  for  simulation  since 
hardware  realizes  the  vector-matrix  multipliers. 

m*****************************************************************  ******* 


HAMMING  DISTANCE  function  looks  at  the  number  of  differences  in  the  actual 
versus  desired  output  vector. 

•A*******************#**#***##**#*#*#*****#******#**#********************/ 


unsigned  integer  hammingjdistancefinteger  pattem_number) 

{ 

Examine  output  vector  and  desired  output  vector  element  by  element  For 
every  nonmatching  output  the  Hamming  number  is  increased  by  one. 

) 


TEST  OUTPUT  function  looks  for  the  test  output  neuron. 
##**••******••****#***•*•*********  *************  **••***•*♦*****•*♦•**  *♦***. 


Like  Hamming  distance  above,  except  uses  test . 
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^tmw«*ww**MI*M<*>**M,t,*,w,>*t>*Hw*w*>*t*M***ll<>t,*>**>>*ww,w 

RANK  HIDDEN1  NEURONS  function  ranks  the  value  of  the  hidden  neurons  in 
ascending  order. 


(unction  rank_hidden  1  _neurons() 

{ 

Ranks  order  neurons  In  Layer  1  using  ranking  algorithm  found  in  numerical  Recipes. 

) 

RANK  HIDDEN2  NEURONS  function  ranks  the  value  of  the  hidden  neurons  in 
ascending  order. 

Same  as  above,  but  for  layer  2  instead. 

y*****************  ****•**•*•****•••***•******•*##**•*#*••***#•  ***#•*#•***# 

SAVE  CURRENT  WEIGHTS1  function  stores  weight  layerl  values. 

•  ••t**«teMS*S*eMS*e*«*ieSM«t*HMSMMSMSSaMtMS**MSase«SMiS*SSS«*^ 

function  save_current_weights1 0 

Saves  weights  from  layer  1  in  temporary  array 


RESTORE  WEIGHTS1  function  restores  weight  layerl  values, 
function  restore_prior_weights10 

Restore  previous  weight  layer  1  values  for  this  particular  training  case. 


SAVE  CURRENT  WEIGHTS2  function  stores  weight  Iayer2  values. 


junction  save_current_weights20 
Same  as  above,  but  for  layer  2 

RESTORE  WEK3HTS2  function  restores  weight  Iayer2  values. 


(unction  restore_prior_weights20 
Same  as  above,  but  for  layer  2 

SAVE  CURRENT  WEIGHTS3  function  stores  weight  Iayer3  values. 


function  save_current_weJghts3() 
same  as  above,  but  for  layer  3 

^»e*********ee**e*  eeeeeeeeeeeeeee*********************  M*****t«**M***«t 

RESTORE  WEIGHTS3  function  restores  weight  Iayer3  values. 


function  restore_prior_weights30 
Same  as  above,  but  for  layer  3 


ADAPT  WEIGHTS  1  function  adapts  one  neuron  at  a  time  until  its  binary  value 
is  inverted. 

............................ . ***"•• . ***•*/ 


function  adapuweightsl  (integer  neuron_number) 

Adapt  neurons  in  layer  1  using  MR  2  rule  criteria 

Uses  singietsl,  Doublets'!,  Triplets  1,  and  Quadiets  1  to  minimize  (train) 

output  error 


I 
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ADAPT  WEIGHT S2  function  adapts  one  neuron  at  a  time  until  its  binary  value 
is  inverted. 

*«*••**•*•**•••*•**•*•«•***•*******•••«•••****«•••*•••**«*«****•••*»«**«*# 


function  adapt_weights2(integer  neuron_number) 
Same  as  above,  but  for  layer  2 

^ae****************************************************** 


SINGLETS  function  adapts  one  neuron  at  a  time.  If  change  results  in 
reducing  Hamming  Distance  then  change  is  kept  Otherwise  weights  are 
restored  to  original  values. 

sss—HeHsesteeseetee— eeteestefHHtHss— —seHtse—esee— eetHti 

function  singlets  (integer  pattem_number) 

/•Rank  analog  neurons  in  order  closest  to  zero*/ 
ranK_hkJden2_neurons(); 

Adapt  in  direction  to  change  corresponding  binary  output 
of  hidden  layer 


Test  for  Hamming  distance  after  each  iteration 


DOUBLETS  function  adapts  two  neurons  at  a  time.  If  change  results  in 
reducing  Hamming  Distance  then  change  is  kept  Otherwise  weights  are 
restored  to  original  values. 


IMSMMASMSMS  SHI  SSSSS  SSSSSf 
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function  doublets(intoger  pattem_number) 
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Same  as  above,  but  adapt  two  neurons  at  a  One 

^MHM*** A*********************************** ft********************** ******* 

TRIPLETS  function  adapts  three  neurons  at  a  time.  If  change  results  in 
reducing  Hamming  Distance  then  change  is  kept  Otherwise  weights  are 
restored  to  original  values. 

function  triplets(integer  pattem_number) 

Same  as  above,  but  adapt  three  neurons  at  a  time 

J************************************************************************* 


QUADLETS  function  adapts  three  neurons  at  a  time.  If  change  results  in 
reducing  Hamming  Distance  then  change  is  kept  Otherwise  weights  are 
restored  to  original  values. 

function  quadlets(integer  pattem_number) 

Same  as  above,  but  adapt  four  neurons  at  a  time. 


ADAPT  WEIGHTS  3  function  adapts  third  layer  neurons. 


function  adapt_weights3(integer  neuron_number) 
Adapts  output  neurons  that  differ  from  target  vector 
Uses  Adapt  Output  Layer  below 


^******SMS*SM*MS*SM*«**S**SMS**S**MtM**SM«ft**«M* 


ADAPT  OUTPUT  LAYER  function  adapts  one  neuron  at  a  time  until  its  binary 
value  is  inverted. 


function  adapt_putpuUayer(integer  pattem_number) 


WRITE  SIM2  WEIGHTS  function  converts  the  weights  and  biases  into  usable 
forms  for  the  optical  simulation  weights 

y»************«*************«e*e*******»«***e*e*****e********e*«**«**e**»*** 

Simulate  LAYER  1  NEURONS  function  calculates  the  weighted  sum  for  each 


neuron  in  layer  1. 


•••••••♦•♦•••a* 


SIMULATE  LAYER  2  NEURONS  function  calculates  the  weighted  sum  for  each 
neuron  in  layer  2. 

^e************************************************************************** 

Madeline  Rule  Two  Algorithm  Training.  The  same  for  optical  hardware 
and  symmetric  MR2  rule  (3layer.c)  so  I  wont  repeat  documentation. 


function  simulateO 
r  Uniform  distribution  of  weights  V 

initialize.weightsO; 


,**•****•***#•***•#•**•********•••*•*•*****••******•*•/ 


/•““Begin  training 
for  (outer Joop^;outerJoop<lteratk>ns;outerJoop++){ 
for  (pattem_oounter=0;pattem_counter<24;pattem_counter++){ 
initiallze.adaptedO; 


/•“““Present  input  vector” 


***•*••*•**•*#***•*•*****«•****•«**«***«**• 


“/ 


binari2e_inputs(Letter_lsb{pattem_counter]P  Letter_msb{pattem_counter]); 
/••••••‘Check  output  vector  and  calculate  Hamming  Distance***************/ 


slmulate.layerl  _neuronsO  I 
simulate Jayer2_neurons(); 


generate_output_neurons(); 


HammDist  *  hanvrTlng_dlstanc8(pattern_counter); 
If  (HanunOist  >  CRITERIA) 

8inglets1  (pattem_counter); 

If  (HanunOist  >  CRITERIA) 
doublets  1  (pattem_counter); 
if  (HanunOist  >  CRITERIA) 
triple  tsl  (pattem_counter); 
if  (HammOist  >  CRITERIA) 
quadletsl  (pattem_counter); 

If  (HammOist  >  CRITERIA) 
singlets2(pattem_counter); 

If  (HammDist  >  CRITERIA) 
doublets2(pattem_counter); 

If  (HammOist  >  CRITERIA) 
triplets2(pattem_counter); 

If  (HammDist  >  CRITERIA) 
quadlets2(pattern_counter); 

If  (HammDist  >  CRITERIA) 
adapt_output_layer(pattem_counter); 

If  (HammDist  >  CRITERIA) 
pattem_leamed[pattern_counter]=0; 
else 

pattem_leamed[pattem_coiinter]=1 ; 


Apply  next  pattern 

Store  weights  In  network  file  and  exit 


save_weighUayersO; 

ytteeeee******************************************************************* 

FILE:  OPTICAL3.C 

DESCRIPTION:  Performs  Madallne  Rule  2  training  with  optical  hardware, 
similar  to  above. 

•*e***«**e*ar***e**********e****«»*ft**e%«e*«ft*»  •**••*•*•*«••*•«••*•*••**••/ 


PROGRAM :  TEST3.C 

DESCRIPTION:  Provides  user  with  a  test  menu  for  the  optical  neural  network. 

«*t******MM**SSt********S«««*St«*S**SMftft***4**S*****ftftS«*S««***«*******«*^ 

TEST  SET  function  looks  for  the  test  output  neuron. 

*M*«t*«*Mt*«MM***M*M**S**t*****t*ft**ft**t**«*M**«t*SH****ft«**MM^ 


unsigned  integer  test_set(integer  test_pattern_number) 
Look  at  output  differences 


TEST  NETWORK  function  test  network  for  given  Hamming  Distance  criteria. 
This  is  the  actual  test  function.  The  two  functions  following  this  one 
are  for  file  i/o. 

ft***************************************************************************/ 


function  test_networkO 

/*****Begin  testing***** . ***** . . 

for  (test_pattem=0;tesLpattem<1 0,1est_pattem++){ 
/•••••••Present  input  vector****4********* . * . 


7 


binarizeJnputs(TesLlsb[test_pattem],Test_msb[test_pattem]); 


Propagate  values  through  alt  three  layers.  Look  at  output 

y . Check  output  vector  *********** . * . . . f 

Apply  next  pattern 


ONE  TEST  function  test  network  for  a  single  Hamming  Distance  criteria. 
MULTIPLE  TESTS  function  tests  network  tor  a  user  prompted  starting  and 
ending  consecutive  Hamming  Distance  criterias. 

_ . . . 

PROGRAM:  DRIVER3.C 

DESCRIPTION:  The  following  routines  provide  hardware  I/O  interface  to 

optical  ANN  and  host  computer.  I/O  is  performed  through  a  96  bit  DIGIO-96 

computer  interface  board  located  at  base  address  HEX  300  utilizing  an 

h-houM  engineered  break-out  board,  and  by  a  46  channel  analog  to  digital 

cooverelon  board  at  base  address  HEX  320.  This  board  reads  32  oplioal  analog 

|  “■tons  corresponding  to  16  optical  outputs  lor  layer  1  and  16  optical 
outputs  for  layer  2. 


•“Wto"  ADC.UYEH10  and  also  layer  2. 

Rc'e' 10  5“li°"  “■  ADC  board  lor  explanation 

. 

i  *•*******••**••*•***#*###*** 

;  S*k,P  Dl9ital  10  ,n,8rface  ftwjl  *  computer,  board  base  address  Is  Hex  300. 
*••••••%•••* 

Unction  MODEJ50 


Writes 


to  sixteen  cascaded  HUGHES  4038  32-pixel  SLM 


waveform  generator  chips. 


Actually  generates  pixel  'ON*  phases  which  get  either  amplified  or  attenuated 
by  programmable  gain  amplifiers. 

function  ENABLE_WEIGHTS() 

function  WRITEJNPUT_VECTOR(unslgned  char  Isb.unsigned  char  msb) 
Simply  output  values  to  omputer  address  ports  (interface) 
outportb(ILEDJ.O,lsb); 
outportb(ILED_HI,msb): 

^********************e****************************************************** 

Write  the  output  from  hidden  layer  as  input  to  output  layer  only  when  using 
all  hardware  thresholding. 

ft***************************************************************************/ 


function  WRITE_HIDDEN1_BINARY0 
similar  to  above,  but  different  addresses. 
outportbflHLEDJ-O,  msb); 
outportb(HLED_HI,lsb); 

Write  the  output  from  hidden  layer  as  input  to  output  layer  when  using 
ADC  board. 

•a**************************************************************************/ 


function  PRESENT_HIDDEN1_VECTOR0 
outportb(HLED_LO,lsb); 
outportb(HLEO_HI,msb); 


This  routine  mites  the  values  of  weights  found  in  WEIGHT_LAYER1[16J[16] 


to  the  corresponding  hardware  programmable  gain  amplifier.  The  programmable 
gain  amplifiers  provides  a  gray  level  driving  signal  to  the  weight  mask. 


function  WRITE_LAYER1_WEIGHTS0 

BOARD1  [32],  r  Board  1  registers  7 

BOARD2[32],  /*  Board  2  registers  V 

BOARD3J32],  /*  Board  3  registers  7 

BOARD4[32],  /*  Board  4  registers  7 

BOARD5[32],  /*  Board  S  registers  7 

BOAR 06(32],  r  Board  6  registers  7 

BOARD7J32],  /*  Board  7  registers  7 

BOARD8(32],  /*  Board  8  registers  7 

0UTWEIGHTS1  [256],  /*  Output  array  for  writing  to  SLMt  V 

DO  =  1 ,  r  Typecasting  numbers  for  bit  shifting  V 

D1  =2, 

D2  =  4, 

D3  =  8, 

D4  =  16, 

D5  =  32, 

D6  =  64, 

D7  a  128; 


Convert  the  weights  and  biases  into  usable  forms  for  the  optical  weights 
Recall  optical  weights  are  limited  in  range. 


Mid  transmission  for  column  16,  will  be  used  for  active  thresholding. 


Optical_layer1[15]Q]  =  90; 


Assign  corresponding  weight  to  slm  driver  board 
/**  Now  set  up  ordinal  serial  sequence  and  concatenate  all  8  **/ 
/**  board  data  lines  to  be  docked  out  *7 


y******«i 


The  following  defines  driver  board  data  line  in  output  port  of  interface 
BOARD  # _ BIT  POSITION _ BIT 


BOARD1  0  1 

BOARD2  1  2 

BOARD3  2  4 

BOARD4  3  8 

BOARD5  4  1b 

BOARD6  5  32 

BOARD7  6  64 

BOARDS  7  128 

I*************************************************************************** 


This  routine  writes  the  values  of  weights  found  in  WEIGHT_LAYER2  [16][16] 
to  the  corresponding  hardware  programmable  gain  amplifier.  The  programmable 
gain  amplifiers  provides  a  gray  level  driving  signal  to  the  weight  mask. 


function  WRITE_LAYER2_WEIGHTS() 
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SimHar  to  above. 


PROGRAM :  TRA1N.C 

DESCRIPTION:  Provides  user  with  a  training  menu. 

^**«**««SiS*««S«S«S««««S««MS««*«M*««****S*fte*St*t**S*S***»*****»»M*t*HSS* 

ONE  RUN  function  trains  network  for  a  single  Hamming  Distance  criteria. 

function  one_runO 

"Enter  Hamming  Distance  Training  Criteria  (between  0  and  3  inclusive) 

MULTIPLE  RUN  function  trains  network  for  a  user  prompted  starting  and 
ending  consecutive  Hamming  Distance  criteria. 

function  multipte_runs{) 

Enter  starting  Hamming  Distance  Training  Criteria  (between  0  and  2  inclusive) 
Enter  ending  Hamming  Distance  Training  Criteria  (between  start  and  3  inclusive) 

TRAIN  MENU  FUNCTION  prompts  user  for  training  options. 
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function  train.menuO 

yweeeeaea «*«•**•*****•****# 

********* 


,Tl 

i« 


FILE:  RANICC 

From  Numerical  Recipes  in  C. 

DESCRIPTION:  Ranks  an  array  of  N  values  in  ascending  order.  Used  in 


MadaOne  Rule  2  Training. 


INDEXX  function  produces  an  array  pointer  which  indexes  the  values  passed 
in  ascending  order. 


A**********************************************************##*##**********#*^ 

function  lndexx(integer  n, double  arrinQ,  Integer  indxQ) 

Similar  to  quicksort 

/****•*****•*****•**********•**•*****••••••*••*»••••••»••*•••#•«••«••»•*••»»« 


RANK  function  takes  the  index  array  and  converts  it  to  a  ordered  rank  array. 


function  Rank(integer  n.integer  indx0.integer  irankQ) 


CHAPTER  6 
Training 

In  multilayer  networks,  it  is  straightforward  to  adapt  the  neurons  in  the  output 
layer  since  network  desired  responses  are  equivalent  to  the  desired  response  of 
the  output  neurons.  Given  the  desired  responses,  the  adaptation  of  the  output 
layer  can  be  realized  with  algorithms  such  as  the  LMS. 

In  layered  nets,  the  fundamental  difficulty  lies  in  obtaining  desired  responses 
for  neurons  in  layers  other  than  the  output  layer.  The  backpropagation  algorithm 
(1,5,1 1,12]  is  a  method  for  establishing  desired  responses  for  'hidden  layer" 
neurons. 

6.1  Algorithm  selection 

Although  backpropagation  is  a  well  known  training  algorithm,  it  is  only  useful  when 
the  nonlinear  activation  i  iunction  is  continuous  and  differentiable  at  all  points.  The 
hardlimiting  threshold  logic  function  is  not  differentiable,  but  is  used  because  it  is  less 
costly  to  implement  in  hardware.  Threshold  activation  functions  can  be  realized  with 
analog  comparators.  Very  few  hardlimiting  function,  multilayer  network  training 
algorithms  have  been  developed  [15,16,20,21,22].  A  multilayer  extension  of  the  LMS, 
known  as  MR2,  was  chosen  for  ease  of  coding[1 6]. 


6.1  MR2  training  rule 

This  training  rule  is  based  on  the  minimum  disturbance  prindple[16].  its  main 
objective  is  to  reduce  the  number  of  output  errors,  or  the  Hamming  Distance  (HD).  The 
HD  is  simply  the  number  of  differences  between  the  actual  output  vector  Yk  and  the 
target  or  desired  vector  Dk,  and  is  expressed  mathematically  as 

HD=X  (Dk-Yjj)  (6.1) 

The  algorithm  is  described  as  follows: 

1 .  Present  an  input  vector  to  the  network. 

2.  Calculate  HD 

3.  If  HD  <  some  criteria  present  next  input  vector,  otherwise  go  to  4. 

4.  Rank  order  neurons.  The  first  layer  neuron  whose  analog  response  is  closest 
to  zero  is  given  a  trial  adaptation  in  the  direction  to  reverse  its  binary  output. 
When  reversal  takes  place,  the  new  output  is  propagated  through  the 
subsequent  layers  and  the  network  produces  a  new  output. 

5.  Calculate  new  HD.  If  new  HD  <  previous  HD,  accept  change,  otherwise  revert 

back  to  previous  weights. 

6.  Adaptively  switch  the  neuron  whose  analog  output  is  next  closest  to  zero.  Go 

to  step  5  until  ail  neurons  in  the  first  layer  have  been  tried. 

7.  If  more  adaptation  is  required,  choose  pairs  of  neurons  in  the  first  layer  and 

adapt  accordingly. 

8.  If  more  adaptation  is  required,  choose  three  neurons  and  adapt  accordingly. 

9.  If  more  adaptation  is  required,  choose  four  neurons  and  adapt  accordingly. 


10.  Go  to  next  layer  and  and  adapt  according  to  steps  4  through  9. 

1 1 .  If  more  adaptation  is  required,  adapt  output  layer.  This  is  straight  forward 
since  desired  output  values  are  known. 

12.  Present  next  input  pattern  and  go  to  step  1 . 

The  source  code  for  this  algorithm  is  found  in  Appendix  4. 


6.3  Training  set 

We  selected  two  letters  from  the  alphabet,  I  and  F,  and  represented  them  with  12 
patterns  each.  The  patterns  have  one  random  pixel  error,  and  also  include  translation 
for  the  letter  I.  These  patterns  are  shown  in  Figure  6.1 . 
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Figure  6.1.  Training  Set 
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6.4  Test  set 

The  network  was  tested  with  10  patterns  not  previously  presented.  Six 
representations  of  the  letter  l  and  4  representations  of  the  letter  F  (Figure  6.2) 
were  presented  to  the  network  after  the  network  was  trained  with  the  training 
set 

•  oo  •  o  •  •  •  o  o  *  o  o  o  • 

•  oo  •  o  o  o  *  o  o  *  o  o  o  * 

•  •  o  •  O  O  0*0  0*0  oo* 

*oo  *00  0*0  O**  o  o  o 

•  OO  #00  0*0  0*0  oo* 

o  •  *  *  *  *  *  *  *  *  *  *  •** 

o  o  *  *  o  o  •  o  o  *  o  •  moo 

°  °  •  •••  •  o  •  '*  •  •  o  •  • 

°  °  •  ooo  •  o  o  *oo  *  o  o 

OO*  *  o  O  •  O  O  *  O  O  *  o  o 

Figure  6.2.  Test  Set 

6.5  Results 

The  network  was  trained  in  two  modes(Figure  6.3) .  Mode  one  is  a  computer 
simulation  of  the  optical  layers  and  incorporates  input  dependent  thresholding.  The  top 
line  in  the  plot  diagram  shows  percent  recognition  vs  hamming  distance  for  the 
computer  simulation.  Mode  2  reads  the  neuron  analog  values  through  an  Analog  to 
Digital  converter  for  rank  ordering  during  training.  For  testing  the  network ,  the 
photodiode  signals  are  input  to  hard  limiting  comparators  which  provide  a  binary  output 
for  the  input  pattern  presented  to  the  optical  hardware.  The  lower  performance  of  the 
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optoelectronic  system  can  be  accounted  for  by  optical  misalignment  of  the  detectors 
and  by  a  number  of  nonfunctioning  pixels  in  each  of  the  SLM’s 


Figure  6.3.  Results 


;  CHAPTER  7 
Conclusions 

I  V.  ' 

\ 

i 

l  It  has  been  shown  that  hybrid  optical  neural  networks  are  very  promising  for  pattern 
I  classification.  The  technology  used  in  the  implementation  of  the  optoelectronic  neural 


I-  network  can  be  incorporated  at  the  Integrated  circuit  level.  This  means  that  fully 
;  interconnected  optoelectronic  networks  of  many  inputs  can  be  designed  and 
’  constructed.  Optics  offer  a  solution  for  the  communication  bottleneck  encountered  by  an 
•  all  electronic  device,  while  electronics  offer  high  gain  and  ease  of  interface  to  current 
i  platforms.  The  MR2  rule  shows  that  electro  optical  ANN  networks  utilizing  hard  limiting 
,  or  threshold  logic  activation  functions  are  trainable .  This  offers  a  significant  alternative 
|  to  using  backpropagation. 

1;  •  I 

\  ■ 

There  is  much  work  to  be  done  in  the  area  of  optical  neural  networks.  Current 
;  technology  can  be  used  to  design  and  demonstrate  an  integrated  optical  chip  device 
\  for  neural  network  computation.  Semiconductor  lasers  and  detectors  are  easily 
\  implementable  at  integrated  circuit  scales.The  crossbar  configuration  is  readily 
;  implemented  at  the  chip  level  in  the  form  of  waveguides  [38, 43, 45].High  density  SLM’s 
t  have  been  demonstrated  at  the  integrated  circuit  level  [30,31 ,32,33,58]  and  can  be 

t 

■  docked  at  MHz  rates  [60].  Everything  is  in  place  to  take  the  findings  of  this  thesis  to  a 

’i 

'  practical  level  and  develop  next  generation  devices.  A  strong  recommendation  is  made 
for  this  work  to  be  carried  further  and  an  optoelectronic  neural  chip  be  manufactured. 


Appendix  1 
LMS  algorithm 

Consider  the  case  of  a  single  neuron.  For  adaptation  the  LMS  ruie  is: 

wk+i  *  Wk  +  jl  C|<Xk  (i) 

lxkl2 

where  Wk+1  is  the  next  value  of  the  weight  vector,  Wk  is  the  present  value  of  the 

weight  vector,  Xk  is  the  present  value  of  the  input  vector,  and  ^  is  the  difference 

\ 

between  the  desired  and  actual  output  before  adaptation.  With  binary  +1  ,-1  values  for 
the  input  vector  elements,  lxkl2  is  equal  to  the  number  of  weights. 

'  ■  ;  I  .  *  •  ;  • 

For  each  adaptation  cycle,  the  above  recursion  is  applied.  This  reduces  the  error  by 
the  fraction  a.  Suppose  that  for  the  k*  iteration  the  error  is: 

ek"dk-XkTWk 

The  error  is  reduced  by  changing  the  weights,  that  is 

Aek  =  A(dk-XkTWk)  =  -XkTAWk 
Using  equation  1 ,  the  weight  change  is 

AWk=  Wk+1-WkS 

lx,!2 

Now  combine  equations  3  and  4  to  obtain: 


(2) 

(3) 

(4) 
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Aek=  -V  SLtyXt 

lxkP 

=  -  V  xk  jjl  £k 
IXkl2 


=  -«*k  (5) 

The  error  is  reduced  by  factor  of  a  as  the  weights  are  changed  while  holding  the 
input  pattern  fixed.  A  new  pattern  then  starts  the  next  adaptation  cycle.  Since  factor 
a  reduces  the  error,  it  must  be  chosen  to  control  stability  and  speed  of 
convergence.  This  value  is  usually  between  1 .0  and  0.1 . 

\ 

i 

F 


[ 

\ 
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Appendix  2 
Schematics 


All  detailed  schematics  were  drawn  with  OrCAD  SDT  III  schematic  layout  tools 
on  a  PC.  Printed  circuit  boards  were  routed  with  OrCAD  PCB  board  design  tools. 

All  printed  circuit  boards,  for  the  exception  of  the  SLM  driver  boards,  were  layed  out 
and  manufactured  at  the  Engineering  Services  Branch  of  Armstrong  Laboratory’s 
Technical  Services  Division.  Eight  major  design  schematics  were  created  and  are 
shown  in  subsequent  pages. 
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LCD  pixel  driver:  DRIVER.SCH 
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Digital  interface  and  SLM  clock:  INTER.SCH 
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Appendix  3 
SPICE  listings 

The  SPICE  simulations  were  performed  with  Microsim's  PSPICE 
on  a  Macintosh  IIFE. 

THRESHOLD  COMPARATOR 

♦WILLIAM  ROBINSON 

♦ANN  PROJECT 

♦ANALYSIS  REQUESTS 

.DC  LIN  VIN  0.5  1.5  0.050 

♦CIRCUIT  COMPONENTS 

R1  1  2  100K 

R2  2  0  100K 

R3  3  4  100K 

R4  4  0  100K 

R5  100  5  5K 

♦SOURCES 

VDD  100  0  DC  5 

VIN  3  0  DC  0.5 

VREF  1  0  DC  1 

* _ , _ ' _ 

♦  connections:  non- inverting  input 
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I  * 


.subckt  LM311 


inverting  input 

positive  power  supply 
|  negative  power  supply 
|  |  open  collector  output 
|  |  |  output  ground 

I  I  II 

1  2  3  4  5  6 


fl 


9  3  vl  1  iee  3  7  dc  100.0E-6  vil  21  1  dc  .45 


E  vi2  22  2  dc  .45  ql  9  21  7  qin  q2  8  22  7  qin 

q3  9  8  4  qmo  q4  8  8  4  qmi  .model  qin 

PNP (Is=800 . OE-18  Bf=500)  .model  qmi  NPN(Is=800 .OE-18 
Bf=1002)  .model  qmo  NPN{Is=800. OE-18  Bf=1000  Cjc=lE-15 
Tr=118 . 8E-9)  el  10  6  9  4  1  vl  10  11  dc  0  q5 

5  11  6  qoc  .model  qoc  NPN{Is=800. OE-18  Bf=34.49E3  Cjc=lE-15 

Tf=364 . 6E-12  Tr=79 .34E-9)  dp  4  3  dx  rp  34 

6.818E3  .model  dx  D(Is=800 . OE-18  Rs=l) 


.ends 


♦COMPARATOR 


XI  4  2  100  0  5  0  LM311 


•OUTPUT  REQUESTS 


.PRINT  DC  V{5) 

.  PROBE 

.END 


Print  data 

VIN 

V(5) 

5 . 000E-01 

6 . 405E-02 

5 . 500E-01 

6 . 405E-02 

6 . 000E-01 

6 . 407E-02 

6 . 500E-01 

6.412E-02 

7 . 000E-01 

6.423E-02 

7.500E-01 

6 . 454E-02 

8 . 000E-01 

6 . 534E-02 

8 . 500E-01 

6.737E-02 

9 . 000E-01 

7.244E-02 

9 . 500E-01 

8 . 563E-02 

1.000E+00 

2 . 454E-01 

1 . 050E+00 

5 . 000E+00 

1 . 100E+00 

5.000E+00 

1.150E+00 

5.000E+00 
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1 . 200E+00 

5 . OOOE+OO 

1 . 250E+00 

5 .  OOOE+OO 

1.300E+00 

5. OOOE+OO 

1 . 350E+00 

5. OOOE+OO 

1 . 400E+00 

5. OOOE+OO 

1 . 450E+00 

5.  OOOE+OO 

1.500E+00 

5. OOOE+OO 

VREF 

V(5) 

5 . 000E-01 

5. OOOE+OO 

5 . 500E-01 

5. OOOE+OO 

6.000E-01 

5. OOOE+OO 

6 . 500E-01 

5. OOOE+OO 

7 . 000E-01 

5. OOOE+OO 

7 . 500E-01 

5. OOOE+OO 

8 . 000E-01 

5. OOOE+OO 

8 . 500E-01 

5. OOOE+OO 

9 . 000E-01 

5. OOOE+OO 

9 .500E-01 

5. OOOE+OO 

l.OOOE+OO 

2.454E-01 

1 . 050E+00 

8 . 563E-02 

1.100E+00  7.244E-02 


1 . 150E+00 
1 . 200E+00 
1 .250E+00 
1 . 300E+00 
1.350E+00 
1 . 400E+00 
1 . 450E+00 
1 . 500E+00 


6 . 737E-02 
6 . 534E-02 
6.454E-02 
6 . 423E-02 
6 . 412E-02 
6 . 407E-02 
6.405E-02 
6 . 405E-02 
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LCD  DRIVER 
♦WILLIAM  ROBINSON 
♦ANN  PROJECT 
♦ANALYSIS  REQUESTS 
.DC  RES  RPOT(R)  40  10K  40 
♦CIRCUIT  COMPONENTS 
R1  100  2  29. 3K 
R2  2  0  100 
R3  3  0  39 
R4  3  4  RPOT  1 
.MODEL  RPOT  RES(R=1) 

♦SOURCES 
VDD  100  0  DC  5 
*LM  324  MODEL 

★  _ _  _ _ _________ _ —  — _ _  _ _ _  _ _ _ _ _  _  _ _ * 


connections:  non- inverting  input 

*  |  inverting  input 

*  |  |  positive  power  supply 

*  Ilf  negative  power  supply 

*  till  output 

*  II  I  I  I  • subckt  LM324  12345 
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* 

cl 

11  12  2.887E-12 

c2 

6 

7  30.00E-12 

dc 

5 

53 

dx 

de 

54  5  dx  dip  90 

91 

dx 

din  92  90  dx 

dp 

4 

3 

dx 

egnd  99  0  poly (2) 

(3 

,0) 

(4,0)  0  .5  .5 

fb 

7 

99 

poly (5)  vb  vc  ve  vlp  vln  0  21.22E6  -20E6  20E6  20E6  -20E6 


ga 

6 

0 

11  12 

188.5E-6 

gem  0  6 

10  99 

59. 

61E-9 

iee 

3  10 

dc 

15. 

09E-6 

hlim  90 

0 

vlim  IK 

qi 

11 

2  13 

qx 

q2 

12 

1 

14  qx 

r2  6 

9 

100. OEp 

rcl 

4 

11  5. 

305E3 

4  12 

5.305E3 

rel 

13 

10  1. 

845E3 

re2  14  10 

1.845E3 

10  99 

13.25E6 

rol 

8 

5  50 

ro2 

7  99  25 

rp  3 

082E3 

vb  9 

0  dc 

0 

vc 

3  53 

dc  1.500  ve  54 

0.65 

vlim  7 

8  dc 

0 

vlp 

91  0 

dc  40  vln 

0  92 

dc  40  .model  dx  D(Is=800.0E-18  Rs=l)  .model  qx 
PNP ( Is=800 . OE-18  Bf =166 . 7 ) 

.  ends 


*0P  AMP 

XI  2  3  100  0  4  LM324 
♦CIRCUIT  INPUT 

♦VPULSE  1  0  PULSE (0  5  . 5US  .1US  . 1US  20US  30US) 
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♦OUTPUT  REQUESTS 


.PRINT  DC  V(4) 

.PROBE 

.END 

Print  data 

R 

V(4) 

4.000E+01 

7 . 401E-02 

8.000E+01 

8.760E-02 

1 . 200E+02 

9 . 988E-02 

1. 600E+02 

1.119E-01 

2.000E+02 

1.243E-01 

2.400E+02 

1 . 375E-01 

i 

2 . 800E+02 

1 . 517E-01 

3 .200E+02 

1 . 669E-01 

3 . 600E+02 

1 . 831E-01 

4.000E+02 

2.000E-01 

4.400E+02  2 . 173E-01 


4 . 800E+02 

5.200E+02 

5 . 600E+02 

6 . 000E+02 

6.400E+02 

6.800E+02 

7 . 200E+02 

7 . 600E+02 

8 . 000E+02 

8 . 400E+02 

8 . 800E+02 

9 . 200E+02 

9.600E+02 

1.000E+03 

1.040E+03 

1.080E+03 

1 . 120E+03 

1.160E+03 

1.200E+03 

1.240E+03 

1.280E+03 

1.320E+03 


2 . 350E-01 

2.528E-01 

2.708E-01 

2 . 888E-01 

3 . 068E-01 

3.249E-01 

3.429E-01 

3 . 610E-01 

3.791E-01 

3 . 971E-01 

4 . 152E-01 

4.332E-01 

4 . 513E-01 

4 . 694E-01 

4 . 874E-01 

5 . 055E-01 

5.236E-01 

5.416E-01 

5.597E-01 

5.778E-01 

5.958E-01 

6.139E-01 


1 . 360E+03 
1.400E+03 
1.440E+03 
1 . 480E+03 
1 . 520E+03 

1 . 560E+03 
1 . 600E+03 
1 . 640E+03 
1 . 680E+03 
1 . 720E+03 
1.800E+03 
1 . 840E+03 
1 . 880E+03 
1 . 920E+03 
1.960E+03 
2 . 000E+03 
2 . 040E+03 
2 . 080E+03 
2 . 120E+03 
2.160E+03 


6 . 320E-01 
6 . 500E-01 
6.681E-01 
6 . 861E-01 
7 • 042E-01 

7 . 223E-01 
7.403E-01 
7.584E-01 
7 .764E-01 
7 . 945E-01 
8.306E-01 
8.487E-01 
8- 667E-01 
8 . 848E-01 
9 . 028E-01 
9.209E-01 
9.390E-01 
9.570E-01 
9.751E-01 
9.931E-01 


2.200E+03  1.011E+00 


2.240E+03  1 . 029E+00 


2.280E+03 

1 . 047E+00 

2.320E+03 

1 . 065E+00 

2 . 360E+03 

1 . 083E+00 

2.400E+03 

1 . 101E+00 

2 . 440E+03 

1.120E+00 

2.480E+03 

1 . 138E+00 

2.520E+03 

1.156E+00 

2 . 560E+03 

1 . 174E+00 

2 . 600E+03 

1.192E+00 

2 . 640E+03 

1 . 210E+00 

2.680E+03 

1.228E+00 

2.720E+03 

1 . 246E+00 

2.760E+03 

1 . 2.64E+00 

2 . 800E+03 

1.282E+00 

2.840E+03 

1.300E+00 

2.880E+03 

1.318E+00 

2.920E+03 

1 . 336E+00 

2 . 960E+03 

1 . 354E+00 

3 . 000E+03 

1.372E+00 

3.040E+03  1.390E+00 


3.080E+03 
3 . 120E+03 
3.160E+03 
3.200E+03 
3 • 240E+03 
3 .280E+03 
3.320E+03 
3.360E+03 

3.400E+03 
3.440E+03 
3 .480E+03 
3.520E+03 
3.560E+03 
3 . 600E+03 
3 . 640E+03 
3.680E+03 
3.720E+03 
3.760E+03 
3.800E+03 
3 .840E+03 


1 . 408E+00 
1 . 426E+00 
1 . 444E+00 
1 .462E+00 
1 . 480E+00 
1.499E+00 
1 . 517E+00 
1 . 535E+00 

1 . 553E+00 
1 . 571E+00 
1.589E+00 
1 . 607E+00 
1 . 625E+00 
1 . 643E+00 
1 . 661E+00 
1 . 679E+00 
1 . 697E+00 
1.715E+00 
1.733E+00 
1 .751E+00 


3 . 880E+03  1.769E+00 
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3 . 920E+03 

1 . 787E+00 

3.960E+03 

1 . 805E+00 

4.000E+03 

1.823E+00 

4.040E+03 

1.841E+00 

4 . 080E+03 

1.859E+00 

l  4.120E+03 

1 . 877E+00 

r  4.160E+03 

l 

1.895E+00 

t  4.200E+03 

t 

1 . 913E+00 

t 

4.240E+03 

i 

1 . 931E+00 

f  4.280E+03 

1.950E+00 

I  4 . 320E+03 

1 . 968E+00 

W  4.360E+03 

1 . 986E+00 

K;  4.400E+03 

m 

2 . 004E+00 

»4.440E+03 

2.022E+00 

J4.480E+03 

2 . 040E+00 

R4.520E+03 

2 . 058E+00 

®4.560E+03 

2.076E+00 

|K.600E+03 

2 . 094E+00 

MB  « 

|fK.640E+03 

2.112E+00 

^®.68°E+03 

V; ' 

2 . 130E+00 

HUBS*  - 

5E720E+03 

2 . 148E+00 

4.760E+03 
4.800E+03 
4.840E+03 
4.880E+03 
4 . 920E+03 
4.960E+03 
5.000E+03 
5.040E+03 
5.080E+03 
5.120E+03 
5.160E+03 

5.200E+03 

5.240E+03 

5.280E+03 

5.320E+03 

5.360E+03 

5.400E+03 

5.440E+03 

5.480E+03 

5.520E+03 

5.560E+03 


2.166E+00 
2.184E+00 
2 . 202E+00 
2 . 220E+00 
2 . 238E+00 
2.256E+00 
2 . 274E+00 
2 . 292E+00 
2.310E+00 
2.328E+00 
2 . 346E+00 

2 . 364E+00 
2 . 382E+00 
2.400E+00 
2.418E+00 
2 .436E+00 
2 . 454E+00 
2 .472E+00 
2 . 490E+00 
2 . 508E+00 
2.526E+00 


5 . 600E+03 
5.640E+03 
5.680E+03 
5.720E+03 
5.760E+03 
5 . 800E+03 
5 . 840E+03 
5.880E+03 
5 . 920E+03 
5 . 960E+03 
6 . 000E+03 
6.040E+03 
6 . 080E+03 
6.120E+03 
6 . 160E+03 
6.200E+03 
6 . 240E+03 
6.280E+03 
6 . 320E+03 
6.360E+03 
6.400E+03 
6.440E+03 


2.544E+00 
2 . 563E+00 
2.581E+00 
2 . 599E+00 
2 . 617E+00 
2.635E+00 
2 . 653E+00 
2 . 671E+00 
2 . 689E+00 
2.707E+00 
2.725E+00 
2.743E+00 
2.761E+00 
2 . 779E+00 
2.797E+00 
2.815E+00 
2.833E+00 
2 . 851E+00 
2.869E+00 
2 . 887E+00 
2 . 905E+00 
2.923E+00 


6.480E+03  2 . 941E+00 


6 . 520E+03 
6.560E+03 
6 . 600E+03 
6 . 640E+03 
6.680E+03 
6 . 720E+03 
6 • 760E+03 
6.800E+03 
6 . 840E+03 
6.880E+03 
6.920E+03 
6 . 960E+03 
7 . 000E+03 

7 . 040E+03 
7.080E+03 
7 . 120E+03 
7 . 160E+03 
7 . 200E+03 
7 . 240E+03 


2.959E+00 
2 . 977E+00 
2 . 995E+00 
3 . 013E+00 
3 . 031E+00 
3 . 049E+00 
3 . 067E+00 
3 . 085E+00 
3 . 103E+00 
3 . 121E+00 
3 . 139E+00 
3 . 157E+00 
3 . 175E+00 

3 . 193E+00 
3 . 211E+00 
3.229E+00 
3.247E+00 
3.265E+00 
3 .283E+00 


7 . 280E+03 
7.320E+03 
7.360E+03 
7.400E+03 
7.440E+03 
7 . 480E+03 
7 . 520E+03 
7 . 560E+03 
7 . 600E+03 
7 . 640E+03 
7 . 680E+03 
7 . 720E+03 
7 . 760E+03 
7 . 800E+03 
7.840E+03 
7 . 880E+03 
7 . 920E+03 
7 . 960E+03 
8 . 000E+03 
8.040E+03 
8 . 080E+03 
8 . 120E+03 


3 . 301E+00 
3.319E+00 
3.337E+00 
3.355E+00 
3.373E+00 
3.391E+00 
3 .409E+00 
3.427E+00 
3 . 445E+00 
3.463E+00 
3 . 481E+00 
3.499E+00 
3 . 517E+00 
3 . 535E+00 
3 . 553E+00 
3 . 571E+00 
3.589E+00 
3 . 607E+00 
3 . 625E+00 
3.643E+00 
3 . 661E+00 
3 . 679E+00 
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8 . 160E+03 
8.200E+03 
8.240E+03 
8 .280E+03 
8.320E+03 
8.360E+03 
8 . 400E+03 
8.440E+03 
8 . 480E+03 
8.520E+03 
8 . 560E+03 
8 . 600E+03 
8 . 640E+03 
8.680E+03 
8.720E+03 
8.760E+03 
8.800E+03 

8 . 840E+03 
8 . 880E+03 
8.920E+03 


3 . 697E+00 
3.715E+00 
3 . 733E+00 
3 . 751E+00 
3.769E+00 
3 . 787E+00 
3 . 805E+00 
3 . 823E+00 
3 . 841E+00 
3 . 858E+00 
3 . 875E+00 
3 . 892E+00 
3 . 907E+00 
3 . 921E+00 
3.933E+00 
3 . 943E+00 
3.951E+00 

3 . 958E+00 
3 . 964E+00 
3 . 970E+00 


108 


8.960E+03 

3 . 974E+00 

9 . 000E+03 

3 . 978E+00 

9.040E+03 

3 . 981E+00 

9 . 080E+03 

3 . 985E+00 

9 . 120E+03 

3 . 987E+00 

9.160E+03 

3.990E+00 

9 . 200E+03 

3 . 992E+00 

9.240E+03 

3.994E+00 

9.280E+03 

3 . 996E+00 

9.320E+03 

3 . 998E+00 

9 . 360E+03 

4 . 000E+00 

9.400E+03 

4 . 002E+00 

9.440E+03 

4.003E+00 

9.480E+03 

4.005E+00 

9.520E+03 

4.006E+00 

9.560E+03 

4.007E+00 

9 . 600E+03 

4 . 008E+00 

9 . 640E+03 

4.010E+00 

9 . 680E+03 

4 . 011E+00 

9.720E+03 

4 . 012E+00 

9.760E+03 

4 . 013E+00 

9 . 800E+03  4.014E+00 
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9.840E+03  4.015E+00 
9.880E+03  4 . 015E+00 
9 . 920E+03  4 . 016E+00 
9.960E+03  4.017E+00 
1 . 000E+04  4 . 018E+00 


•  l 


\ 


PHOTOVOLTAIC  RECEIVER 


♦WILLIAM  ROBINSON 

*ANN  PROJECT 

♦ANALYSIS  REQUESTS 

.DC  LIN  VIN  0.050  0.350  0.025 

♦CIRCUIT  COMPONENTS 

R1  1  2  909 

R2  2  0  1MEG 

R3  3  0  909 

R4  3  4  5K 

♦SOURCES 

VDD  100  0  DC  5 

VIN  1  0  DC  0.25 

♦LM  324  MODEL 


connections:  non- inverting  input 

*  |  inverting  input 

*  |  |  positive  power  supply 

*  III  negative  power  supply 

*  I  I  I  I  output 


.subckt  LM324  12345 

* 

Cl  11  12  2 . 887E-12 
c2  67  30.00E-12 

dc  5  53  dx 

de  54  5  dx 

dip  90  91  dx 

din  92  90  dx 

dp  4  3  dx 

egnd  99  0  poly(2)  (3,0)  (4,0)  0  .5  .5 

fb  7  99  poly (5)  vb  vc  ve  vlp  vln  0  21.22E6  -20E6  20E6 

20E6  -20E6 

ga  6  0  11  12  188.5E-6 

gem  0  6  10  99  59.61E-9 

iee  3  10  dc  15.09E-6 

hlim  90  0  vlim  IK 

ql  11  2  13  qx 

q2  12  1  14  qx 

r2  69  100. 0E3 

rcl  4  11  5.305E3 

rc2  4  12  5.305E3 


rel  13  10  1.845E3 
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re2 

14 

10 

l.l 

B45E3 

ree 

10 

99 

13 

.  25E6 

rol 

8 

5 

50 

ro2 

7 

99 

25 

rp 

3 

4 

9.1 

082E3 

vb 

9 

0 

dc 

0 

VC 

3 

53 

dc 

1.500 

ve 

54 

4 

dc 

0.65 

vlim 

7 

8 

dc 

0 

vlp 

91 

0 

dc 

40 

vln 

0 

92 

dc 

40 

.model  dx  D(Is=800.0E-18  Rs=l) 

.model  qx  PNP(Is=800.0E-18  Bf=166.7) 
.ends 


*0P  AMP 

XI  2  3  100  0  4  LM324 
♦OUTPUT  REQUESTS 
.PRINT  DC  V(4) 

. PROBE 


END 
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Print  data 

VIN 

V(4) 

5.000E-02 

3.287E-01 

7 . 500E-02 

4 . 910E-01 

1 . 000E-01 

6.533E-01 

1.250E-01 

8.156E-01 

1 . 500E-01 

9 . 779E-01 

1.750E-01 

1 . 140E+00 

2.000E-01 

1.303E+00 

2 . 250E-01 

1 . 465E+00 

2 . 500E-01 

• 

1.627E+00 

2.750E-01 

1.789E+00 

3.000E-01 

1 . 952E+00 

3.250E-01 

2.114E+00 

3.500E-01  2.276E+00 


DFQ2  CIRCUIT 
♦WILLIAM  ROBINSON 
♦ANN  PROJECT 
♦ANALYSIS  REQUEST 
.DC  VIN  -15  15  .5 
♦RESISTORS 
R1  1  5  10K 
R2  2  4  10K 
R3  4  5  15K 
R4  5  8  15K 
R5  1  2  10K 
R6  20  2  10K 
R7  6  8  10K 
R8  1  6  10K 
R9  21  6  10K 
RF  5  10  10K 
RIO  1  11  10K 
Rll  11  22  10K 
R12  11  13  10K 
R13  1  14  10K 


R14  14  23  10K 
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R15  14  16  10K 
R16  13  5  25K 
R17  16  5  50K 
♦DIODES 

D1  2  3  D1N4154 
D2  3  4  D1N4154 
D3  7  6  D1N4154 
D4  8  7  D1N4154 
D5  11  12  D1N4154 
D6  12  13  D1N4154 
D7  15  14  D1N4154 
D8  16  15  D1N4154 
*OP  AMPS 

XI  0  2  30  31  3  LM324 
X2  0  6  30  31  7  LM324 
X3  0  5  30  31  10  LM32 
X4  0  11  30  31  12  LM324 

X5  0  14  30  31  15  LM324 

♦SOURCES 
VI  20  0  DC  5 

V2  0  21  DC  5 


V3  22  0  DC  10 
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V4  0  23  DC  10 
VCC  30  0  DC  15 
VDD  0  31  DC  15 
VIN  1  0  DC  1 

.model  D1N4154  D(Is=0.1p  Rs=4  CJO=2p  Tt=3n  Bv=60  lbv=0 . Ip) 

*OP  AMP  SUBCIRCUIT 

*  connections:  non- inverting  input 

*  |  inverting  input 

*  |  J  positive  power  supply 

*  *  I  I  I  negative  power  supply 

*  I  I  I  I  output 


.subckt  OP-07  12345 

* 


cl 

11  12  8. 

661E-12 

c2 

6  7  30 . 00E-12 

dc 

5 

53 

dx 

de 

54  5  dx 

dip  90 

91 

dx  din  92  90  dx 

dp 

4 

3 

dx 

egnd  99 

0  poly (2) 

(3, 

0)  (4,0)  0  .5  .5 

fb 

7 

99 

poly (5)  vb  vc  ve  vlp  vln  0  221. 0E6  -200E6  200E6  200E6  -200E6 


ga 

6 

0  11  12.  113.1E-6 

gem  0  6 

10  99 

56.69E-12  iee 

10 

4  dc 

6.002E-6  hlim  90 

0  vlim  IK 

qi 

11  2  13  qx 

q2 

12 

1  14  qx  r2  6 

9  100. 0E3 

rcl 

3  11  8.841E3 

rc2  3  12  8.841E3  rel  13  10  219.4  re2  14  10  219.4 


ree 

10  99  33.32E6 

rol 

8 

5  40 

ro2  7  99 

20 

rp 

3 

4  12 

.  03E3  vb  9 

0  dc  0 

vc 

3  53  dc  1 

ve 

54 

4 

dc  1 

vlim  7  8  dc 

0 

vlp 

91  0 

dc  30  vln 

0 

92  dc  30 

.model  dx  D(Is=800.0E-18  Rs=l)  .model  qx  NPN(Is=800.0E-18 
Bf =3 . 000E3) 

.ends 


*  connections:  non- inverting  input 

*  J  inverting  input 

*  1  |  positive  power  supply 

*  ill  negative  power  supply 

*  till  output 


.subckt  LM324  12345 


* 

cl 

11  12  2. 

887E-12 

c2 

6  7  30 . 00E-12 

dc 

5 

53 

dx 

de 

54  5  dx 

dip  90 

91 

dx  din  92  90  dx 

dp 

4 

3 

dx 

egnd  99 

0  poly{2) 

(3, 

0)  (4,0)  0  .5  .5 

fb 

7 

99 

poly(5)  vb  vc  ve  vlp  vln  0  21.22E6  -20E6  20E6  20E6  -20E6 
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3  10 

dc 

15.09E-6 

hlim 

90 

0 

vlim  IK 

qi 

11 

2 

13  qx 

Q2 

12 

1  14  qx 

r2 

6 

9 

100. 0E3 

rcl 

4 

11 

5.305E3 

rc2 

4  12 

5.305E3 

rel 

13 

10  1.845E3 

re2  14  10 

1.845E3 

ree 

10  99 

13.25E6 

rol 

8 

5  50  ro2 

7  99  25 

rp  3 

4  9. 

082E3 

vb  9 

0  dc 

0 

vc  3  53 

dc  1.500  ve  54 

4  dc 

0.65 

vlim  7 

8  dc 

0 

vlp  91  0 

dc  40  vln 

0  92 

dc  40  .model  dx  D(Is=800 . OE-18  Rs=l)  .model  qx 
PNP (Is=800 . OE-18  Bf=166 .7) 

.ends 


* OUTPUT 
PRINT  DC  V(10) 


. PROBE 

.END 

Print  data 

VIN 

V(10) 

-1 .500E+01 

6.333E+00 

-1.450E+01 

6.367E+00 

-1.400E+01 

6.400E+00 

-1.350E+01 

6.433E+00 

-1.300E+01 

6.467E+00 

-1.250E+01 

6.500E+00 
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-1.200E+01 

6 . 533E+00 

-1.150E+01 

6.567E+00 

-1 . 100E+01 

6 . 600E+00 

-1.050E+01 

6 . 633E+00 

-1.000E+01 

6 . 666E+00 

-9.500E+00 

6 . 500E+00 

-9.000E+00 

6.333E+00 

-8.500E+00 

6.166E+00 

-8.000E+00 

6. OOOE+OO 

-7 . 500E+00 

5.833E+00 

-7 .OOOE+OO 

5 . 666E+00 

-6.500E+00 

5 . 500E+00 

~6. OOOE+OO 

5 . 333E+00 

-5.500E+00 

5 . 166E+00 

-5. OOOE+OO 

4 . 999E+00 

-4.500E+00 

4.499E+00 

-4. OOOE+OO 

3.999E+00 

-3.500E+00 

3.499E+00 

-3. OOOE+OO 

2.999E+00 

-2 .500E+00 

2.499E+00 

-2. OOOE+OO  1.999E+00 


-1 . 500E+00 

-l.OOOE+OO 

-5.000E-01 

O.OOOE+OO 

5.000E-01 

l.OOOE+OO 

1 . 500E+00 

2.000E+00 

2.500E+00 

3.000E+00 

3.500E+00 

4.000E+00 

4.500E+00 

5.000E+00 

5.500E+00 

6.000E+00 

6.500E+00 

7.000E+00 

7.500E+00 

8.000E+00 

8.500E+00 


1.499E+00 
9.991E-01 
4.991E-01 
-8.353E-04 
-5.008E-01 
-1 . OOIE+OO 
-1 . 501E+00 

-2 . OOIE+OO 

-2 . 501E+00 
% 

-3. OOIE+OO 
-3 . 501E+00 
-4. OOIE+OO 
-4 . 501E+00 
-5.000E+00 
-5 . 167E+00 
-5 . 333E+00 
-5 . 500E+00 
-5 . 667E+00 
-5.833E+00 
-6.000E+00 
-6 . 167E+00 


9.000E+00 


-6.333E+00 


9.500E+00 

1.000E+01 

1 . 050E+01 

1.100E+01 

1.150E+01 

1.200E+01 

1.250E+01 

1 . 300E+01 

1 . 350E+01 

1.400E+01 

1.450E+01 

1.500E+01 


-6 . 500E+00 

-6 . 667E+00 

-6.733E+00 

-6.800E+00 

-6.867E+00 

-6 . 933E+00 

-7 . OOOE+OO 

-7 . 067E+00 

-1 . 133E+00 
• 

-7.200E+00 
-7.267E+00 
-7 . 333E+00 
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Appendix  4 

Source  code  listing 

/ 

Overview 

The  optical  hardware  is  controlled  by  a  PC  compatible  computer  and  two 
interface  boards.  Much  software  was  developed  not  only  to  control  the  hardware, 
but  also  to  train  with  the  MR2  rule.  To  run  the  system,  type  MENU  at  the  user’s 
prompt  and  select  the  available  choices. 

The  source  code  listings  are  well  documented  and  provide  a  good  explanation  for  ail 
the  functions.  The  system  software  consists  of  12  separate  files  listed  below  followed  by 
a  brief  description.  Uniform.c  contains  functions  for  uniformly  distributing  network 
weights  during  weight  initialization.  Ftankx  contains  function  for  array  indexing  and 
ranking.  Network3.c  saves  and  recalls  network  weights  for  training  and  testing  of 
patterns.  File.c  provide  file  input  and  output  capability.  Train.c  provides  a  menu  for 
training  the  network.  Test3.c  tests  patterns  after  network  has  been  trained.  Share3.c 
provides  MR2  training  rule  functions  shared  by  optical  system  software,  optical 
simulation  software,  and  by  traditional  MR2  algorithm  training.  Driver3.c  provides 
hardware  digital  and  analog  input  and  output  functions.  Optical3.c ,  Sim3_d.c  and 
3layer.c  are  the  actual  MR2  training  software  for  the  three  systems. 


123 


Appendix  4 

Source  code  listing 

Overview 

The  optical  hardware  is  controlled  by  a  PC  compatible  computer  and  two 
interface  boards.  Much  software  was  developed  not  only  to  control  the  hardware, 
but  also  to  train  with  the  MR2  rule.  To  run  the  system,  type  MENU  at  the  user's 
prompt  and  select  the  available  choices. 

The  source  code  listings  are  well  documented  and  provide  a  good  explanation  for  all 
the  functions.  The  system  software  consists  of  12  separate  files  listed  below  followed  by 
a  brief  description.  Uniform.c  contains  functions  for  uniformly  distributing  network 
weights  during  weight  initialization.  Rankx  contains  function  for  array  indexing  and 
ranking.  Network3.c  saves  and  recalls  network  weights  for  training  and  tpsting  of 
patterns.  FHe.c  provide  file  input  and  output  capability.  Train.c  provides  a  menu  for 
training  the  network.  Test3.c  tests  patterns  after  network  has  been  trained.  Share3.c 
provides  MR2  training  rule  functions  shared  by  optical  system  software,  optical 
simulation  software,  and  by  traditional  MR2  algorithm  training.  Driver3.c  provides 
hardware  digital  and  analog  input  and  output  functions.  Optica!3.c  ,  Sim3_d.c  and 
3!ayer.c  are  the  actual  MR2  training  software  for  the  three  systems. 


PROGRAM :  MENU3.C 
WRITTEN  BY:  Wiliam  Robinson 
CLASS:  Master's  Thesis 

DESCRIPTION:  Provides  simple  menu  interface  for  Optical  ANN.  Allows  user 
to  select  various  modes  of  training. 


**•*****#«•*« **#* 


f  include  Tndude3.c* 

SYSTEM  PARAMETERS  function  asks  user  for  choice  of  system  (Optical  or 
Software  only)  and  forweight  distribution  bounds. 

void  system_parameters(void) 

{ 

Intnum.numl; 

drscrO; 
gotoxy(1 ,5); 

printf(aPlease  enter  Initial  weight  uniform  distribution  parameters*); 

printffNn  Lower  Bound :  *); 

scanff%da,&Mtn); 

printfCNn  Upper  Bound :  *); 

scanf(*%d*,&Max); 

printf(*\n  Please  enter  number  of  Iterations  for  training  set  (ie  3):*); 

scanf(*%d*,&lterations); 

drscrO; 

do{ 

printffPlease  select  1:  Optical  or  2:  Software  Only :  *); 
scanf(*%d*,&num); 

}  while  ((num  <  1)  I  (num  >  2)); 
if(num=1M 
OPTICAL  =1; 

SOFTWARE  =0; 

) 

else{ 

SOFTWARE  =1; 

OPTICAL  =0; 

dot 

printf("\n  Please  select  training  option  *); 
printfCNn  1.  Normal  MR2  training  *); 
printfCNn  2.  Optical  hardware  simulation  "); 
printfCNn  Enter  option  selected:  *); 
scanf("%d",&num1); 

)  while  ((numi  <  1)  I  (numl  >  2)); 

SOFTWARE_VERSION  =  numl; 

) 

system_8elected  *  1; 

) 


A****************************** 


Main  Program 

***..**. . * - **** *  * 

■WWWWnWlwVWVWV 

int  main(void) 

t 

int  choice; 
get_lt=0; 


saveJfcxO; 

OPT1CAUO; 

SOFTWARES 

system_setected=0; 

Display  the  ANN  Menu 


do{ 

drscrO; 

printf(-Nn-); 

printf  CMMOpticaJ  ANN  Main  MenuVi'); 
printf  C  \n*); 

printf  (*MO.  Quito*); 

printf  ("M 1.  System  parameters  \n*); 
printf  CM  2.  File  menuW); 
printf  CM  3.  Train  memto*); 
printf  CM  4.  Test  memAn*); 
printf  CM->"); 

scanfC%d*,&choiC8); 
switch  (choice)  { 
case  0:  exit(O); 
break; 

case  1:  system_parameters(); 
break; 

case  2:  fiJe_menu(); 
break; 

case  3:  tralnjmenuO; 
break; 

case4:tesLmenuO; 

break; 

defaultbreak; 

} 


}  while  (choice  1=  0); 
return  0; 

) 

/*•******“******“******“**•“***********•*******“****•**“*“********“ 

FILE:  Indude3.c 

WRITTEN  BY:  WILLIAM  ROBINSON 

DESCRIPTION:  Contains  headers,  definitions,  and  global  variables  used  In  the 
optical  neural  network  system  simulation. 


Standard  include  ffles 

MM«SM*MM*H***4«M*HM*M*ft***ftM*S*SSSHMSMH*MH***SM**ftM*tSSt*^ 


findude  <stdo.h> 
findude  <stdOb.h> 
findude  <math.h> 


findude  <conk>.h> 
findude  <time  Jo 
findude  <dos.h> 
findude  <string.h> 
findude  <ctype.h> 

y*#****«Q£p||yj|yjQ|^g*4****«********e**********«********«***e*#«*****«*****4y 
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fdeflne  ADC_BASE  0x0320 

fdeflne  TRIGGER  0 

KMne  SET.GAIN  (ADC_BASE  +  3) 
fdeflne  GAIN  3 

fdeflne  ADC.CONVERT  (ADC.BASE  +  1)  /*  12  BIT  CONVERSION  7 

fdeflne  SET.CHANNEL  (ADC.BASE  +  2) 

fdeflne  READ_LOW  AOC.BASE 

fdeflne  READlHIGH  (ADC.BASE  +  1} 

fdeflne  HOC  {ADC_BASE  +  2) 

fdeflne  ADC JBUSY  0x7F 

/**Ht>*DtGITAIi  I/O  BOARD*************************************************/ 

fdeflne  BASE  0x300  /*  Interface  board  base  address  V 

fdeflne  SSBIT  0x00  r  Wiper  Stack  Select  Bit  7 

fdeflne  ILEDJ.0  0x301  /"Base  +  1  PortC*/ 

fdeflne ILED.HI  0x302  /*Base  +  2  PortB*/ 

fdeflne  HLED_LO  0x300  /"Base  +  0  Port  A  V 

fdeflne HLED_HI  0x306  r Base  +  6  PortC*/ 

fdeflne  HOUT.LO  Qx30A  r  Base  + 10  Port  C  7 

fdeflne  HOUT_HI  0x309  /"Base +  9  PortB*/ 

fdeflne  H20UTJ.0  0x306  /"Base +  8  Port  A"/ 

fdeflne  H20UT.HI  0x30E  r Base +14  PortC*/ 

fdeflne  LCD1 JDATA  1  T  Write  to  Base  +  5  Port  B0  V 

fdeflne  LCOI.LOAD  2  /*  Write  to  Base  +  5  POrtBI  7 

fdeflne  LCD1.CLOCK  1  /*  Write  to  Base  +  4  Port  A0  */ 

fdeflne  LC02.DATA  .4  /*  Write  to  Base  ♦  5  Port  B2  */ 

fdeflne  LCD2J-OAD  8  /*  Write  to  Base  +  5  PortB37 

fdeflne  LCD2JCLOCK  4  /*  Write  to  Base  +  4  Port  A2  7 

fdeflne  LAYER1 JLOAO  32  /*  Write  to  Base  +  5  PdrtBSY 

fdeflne  LAYER1  jCLOCK  2  T  Write  to  Base  +  4  Port  A1  7 

fdeflne  LAYER2_LOAD  128  /*  Write  to  Base  +  5  PortB7*/ 

fdeflne  LAYER2_CLOCK  8  /*  Write  to  Base  +  4  Port  A3  7 


/*** — 48  CHANNEL  ANALOG  INPUT  BOARD* 


Function  prototypes 


PUNIFORM.C*/ 


double  u32(void); 

double  untform_cfist(double  a,  double  b); 
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/"RANK.C7 

void  lndexx(lnt  redouble  arrinQ,int  IndxQ); 
void  Rank0nt  n,lnt  IndxQ.int  IrankQ); 


/*DRIVER3.C*/ 

void  ADC_LAYER1  (void); 

void  ADCJLAYER2(vokJ); 

void  MODEJKvoid); 

void  ENABLE_WEK3HTS<void); 

void  WRITE_INPUT_VECTOR(un8lgnecl  char  Isb, unsigned  char  msb); 

void  PRESENT JHIDOEN1_VECTOR(void); 

void  WRITE_HIDDEN1_BINARY(void}; 

void  WRlTE_LAYER1_WEIGHTS{void); 

void  WRJTE_LAYER2_WEK3HTS<void); 

void  LEDS_OFF  (void); 


fSHARE3.C*/ 
voidfatftialze_vieights(void); 
void  biMalze_adapted(void); 

void  blnart£ejnputs(unsignied  char  Isb, unsigned  char  msb); 

Void  generete_output_neurons(void); 

void  generate_neurons(void); 

unsigned  bit  hammlng_dtetaiice(int  pattem_number); 

unsigned  bit  te*C_output(lnt  tesLpattern.number); 

void  ranK_hkJden1_neurons(void); 

void  ranK_hidden2_neurons(void); 

void  save_current_weights1  (void); 

void  sav»_current_weights2(void); 

void  save_current.weights3(void); 

void  restore_prior_weights1  (void); 

void  restore_prior_weights2(void); 

void  restore_priorjweights3(void); 

void  sbigiets1(bit  pattem_number); 

void  doubietsl  (bit  pattem.number); 

void  tripletsl  (Int  petlem_number); 

void  quatfetsl  (bit  pattem_number); 

void  sbigiets2(lnt  pattem.number); 

void  doubiets2(bit  pattem_number); 

void  tripiets2(int  pattem_number); 

void  quadtots2(bit  pattem_pumber); 

void  adapLoutpuUayer(bit  pattem_number); 

void  aciapt_we*ghts  1  (int  neuronjnumber); 

void  adapLweights2(bit  neuron.number); 

void  adapt_weights3(tait  neuron_number); 

/•OPTICAL**/ 

void  readJUdden  1  _binary(void); 
void  read.hidden2j)biaiy(void); 
void  read_hidden1_analog(void); 
void  read_Wdden2_anaiog(void); 
void  comparejiidden1_neurons(void); 
void  compare Jddden?_neurons(voU}; 
void  opdcaLann(void); 

/*SIM3_D.C*/ 

double  dynamfc_threshotd(int  biput(16D; 
void  Write_sbn1_weights(void); 
void  Wrtte_sim2Lvieights(void); 
void  sbnuiate_layer1_newonsj^oid); 
void  simulate Jayer2_neurons(voki); 
void  sbnuiate(void); 

/•3LAYER.C7 

void  generate Jayer1_neuroos( void); 
void  generate Jayer2_neurons(void); 
void  normaUrabdng(void); 

fNETWORK3,C  7 

tatsave_«veighUayer8(void); 

brtget_weightJayers(void); 


/•RLE.C*/ 
void  geUle(void); 
void  save_»e(void); 
void  f*e_menu{void); 

/TRAIN.C*/ 
void  ooe_run<void); 
void  muitipie_runs(void); 
void  train_menu<voW); 

/TEST3.C*/ 

unsigned  Int  test_set(lnt  test_pattem_number); 
void  tesUwtwork(void); 
void  one_test(void); 
void  muMple.testsi^oid); 
void  test_memj(voJd); 

rMENU3_D.C*/ 

void  sy8tem_parametefs(vofcQ; 

Global  variables 

double  Min-60; 

double  Max-eO;  /*For  weights  uniform  (Sstribution*/ 


int  Nflag*  1; 

r  lor  random  number  generator  */ 

FILE  •kL.flle. 

*out_file;  r  file  pointers  for  network  weights  7 

double  Seedl  *  12345.0,  r  initial  seed  for  the  random  number  gen  V 

Seed2  ■  67890.0;  /*  initial  seed  for  the  random  number  gen  V 

char  in_name(1 2].out_name(1 2] ; 

Training  Vectors  are  hardcoded  in  alpha  arrays  for  easier  coclng. 

The  following  are  global  arrays. 

t  Input  Vectors  V 
unsigned  char 
/*  Hardware  input  vectors  V 

Letler_«sb(24HOx1  F.OxI  D,Qx9F,0x3F,0x1 7,OxAO,OxE8,QxEO,OxEO,QxOO, 
Qx00,Qx08IQxBF,QxBF,0x9F,0xAF,0xBE,0xBF,Qx3F,QxFF,QxBF,  OxBF.OxBF.OxBD}, 
Letter_msb|24H0xa0,0x83, 0x80,0x80,0x80, 0x80, 0x83,0x83, 0x88, 0xF4, 
0xFC.0xFC,0x94,0x84, 0x94, 0x94, 0x94, 0x90,0x94, 0x94, 0x96, 0xD4,0xB4, 0x04}, 


t*  Target  output  vectors  V 

TesU«b(10WQx9F,  0x1  F,0xE1,0xE0, 0x00, 0x20,  QxB7,0x3F,0xBF,0xBB}, 
Test_msb(1 0]={0x80, 0x82, 0x83, 0xA3, Ox  DC, OxFC, 0x94, 0x94 ,0x9C, 0x94); 


r  Target  output  vectors  7 

unsigned  W  Uttectaroe«24H1,1.1.l.1.1.1,1.1.1.1,1^A2^^^^^}; 
unsigned  int  Letter.testf  10}-{1 .1,1,1 .1 ,1 2222)’, 
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Int  lnput_pattem[16];  /*  +1,-1,  or  0  binary  input*  V 
int  CRITERIA;  r  Hamming  dtetance  requirement  V 

doubt*  Hkkfen1_analog_vaJue(1 6); 
int  HJdden1_binary_vaJue(16): 

Int  Wdden1_ranK_vaJue(161; 

Int  HkJdenl  j*daptad(1<&  /*tf  binary  value  has  been  inverted  7 

double  HMden2_anatog_vaJue{1 6}; 

Int  Mdden2Jrinaryjtf*iue(16]; 

Int  HWden2_ranK_value(1 6]; 

Int  HkJden3_adapted(16];  /*•  binary  value  has  been  inverted  V 

double  Output_analog_value{5); 

unsigned  Int  Output_blnaiy_vahje(5]; 

int  Output_adapted[5]; 

int  HammDist;  /*  Hamming  dstance  7 

nt  pattenr_leamed(24]; 

int  Iterations; 

lntget.lt; 

Int  save  Jt; 
intsystenuseiected; 

Int  OPTICAL; 

kit  SOFTWARE; 

int  SOFTWARE.VERSION; 

double  THRESHOLD1; 

double  THRESHOLDS 

double 

Welght_layer1  [1611 6],  r  Hold*  computer  weights  7 

WeighUayer2{16J161, 

WWghUtayerSlieKSJ. 

TempJayer1[16IU 
TempJayer2l1S|16], 

Tempjayer3(1«pi, 

Slm.layeirt  [16(16]. 

SlmJayer2[16I16]; 
unsigned  char 

OpdcaUayer1[16Kl6]. 

OpdcalJayer2l1 6J1 6], 

ThreshokLcolumn; 


r  Temporary  weight  storage  7 

A  For  storing  simulated  weights  7 

r  External  hardware  optical  weights  7 
/"Used  for  dynamic  Thresholding  7 


•Include  "untfomtc" 
•include 'rank-c* 
•include  "networfca.c” 
•include  mex* 
•include  "train-c? 
•include  testae' 
•include  *share3x* 
•include 'drtver&c* 
•indude  "opdouac 
•include  'slm3_d.c‘ 
•include  *3iayer.c' 


e**#***ee***e 


F1LE:share3.c 


WRITTEN  BY:  WMam  Robinson 

DESCRIPTION:  Contains  functions  shared  by  simulation  and  optical 
hardware  during  training  phase. 


INITIALIZE  WEIGHTS  function  uniformly  distributes  weight  values  in  each 
of  the  weight  layers. 


void  toitiatoe_weights(voJd) 

{ 

mtij: 

for  (M);k16,i++) 

forQ-O^ie^-K 

uniform_tfcrt(Mln,Max); 

Weight_iayer20IQ  *  unifom\_cflst(Mln.Max); 


for  (W0;k16;i++) 
for  0»O;J<3j++) 
Weight_teyer3{iID  = 


uniform_(tist(MinlMax): 


SMMSMSSSMtS 


INITIALIZE  adapted  function  clears  hidden  neuron  adaptedvatoe. 

void  foWabe_adapted(void) 

{ 

inti; 

for  (k0;k16;l++){ 

HkkJen1_adapted[i)  =  0; 

HkJden2_adapted(i]  s  0; 


BINARIZEJNPUTS  function  reads  msb  and  Isb  input  vector  values  and  assigns 
a  correspondng  +1  or  0.  No  symmetry  is  used.  This  Is  only  used  to 
calculate  hidden  neuron  analog  values. 
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temp(15)>1;  /*  Bias  input  set  to  +1  */ 

If  (SOFTWARE_VERSION»“1X 


for  (number»0;number<1 6;number  ++) 

If  (temp(number])  lnput_pattem[number]  =  1; 
else  lnpuLpattem(nurnber}=-1; 

} 

etaw{ 

for  (number=0;number<1 6;number  ++) 

If  (temp{number])  lnput_pattem[number]  =  1; 
else  lnpuLpattern{number>=0; 

1 


GENERATE  OUTPUT  NEURONS  function  calculates  the  weighted  sum  for  each 
output  neuron. 


void  generate_output_neurons(voJd) 

{ 

Int  number 
Int  column; 

/"initialize  values*/ 
for  (numbera0;number<3;number++) 
Output_anatog_value[number]=0; 
for  (numberfe0;number<3;nuinber++){ 
for  (column=0;column<1 6;column++) 
Output_analog_value{number]  += 

Hkkien2_blnary_value{coiumn]  *  WeighL>ayer3[column][number]; 

for  (number=0;number<3;number++){ 
if  (Output_analog_value[number]  >=0) 
Output_Wnary_value[number]  =  1; 
else 


Output_blnary_value[number]  =  0; 


GENERATE  NEURONS  function  generates  neurons  for  all  3  layers  independent 
of  configuration. 

void  generate_neurons(void) 

{ 

If  (0PTICAL=1X 

compare_hidden1_neurons0; 

PRESENT_HIDDEN1_VECTOR0; 

compare_hldden2_neurons0; 

generatejoutpuLneuronsO; 

} 

else  If  (SOFTWARE_VERSION=2){ 

8kmilate_layer1_neurons0; 
sJmulate_layer2_neuronsO; 
generate_output_neuronsO ; 

} 


else{ 


t. 
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generate Jayer1_neurons<); 
generate Jayer2_neurons(); 
generate joutpuLneuronsO; 
) 


HAMMINQ  DISTANCE  function  looks  at  the  number  of  deferences  In  the  actual 
versus  desired  output  vector. 

Net HSH»H—eeeeH—eeeeeeeeese^ 

unsigned  Int  hammlng_dtetance(bit  pattern_number) 

{ 

unsigned  bit  actuaLvector; 
unsigned  bit  differences; 
unsigned  bit  ax_or_vector; 
bit  number. 


actuaLvector  -  (Output_binary_value{0]  *  1)  +  (Output_binafy_vakie{1)  *  2)  + 
(Ou1puLbbiary_vaJue{2]  *  4); 

ex_or_vector  a  actuaLvector A  Letter_target{pattem_number]; 
differences  =  (ex_or_vector  &  1)  +  ((ex_or_vector  &  2)  »1)  + 

((ex_or_vector  &  4) »  2); 
retum(dHferences); 

f  } _ _ _ _ _ 

TEST  OUTPUT  function  loqks  for  die  test  output  neuron. 

'  unsigned  bit  tesLoutput(int  tesLpattem_number) 

i  t 

I  unsigned  bit  actuaLvector; 

I  unsigned  bit  differences; 

|  unsigned  bit  and_vector, 

r,  actuaLvector  =  (Output_binary_value{0]  *  1)  +  (OutpuLWnary_value[1  ]  *  2)  + 
f’  (OutpuLbinary_value[2J  *  4); 

f  anrLvector  =  actuaLvector  &  Letter  jarget{tesLpattern_number]; 

S  differences  =  (and_vector  &  1)  +  ((and_vector  &  2)  »1)  + 

((andLvector  &  4) »  2); 
retum(differences); 
l  1 

^  yee *•#****#•*** *#e*ee**e#**«eeeeee*ee*ee*eee**eee#*e*#eee**e#**ee*e***ee**#** 

K  RANK  HIDDEN1  NEURONS  function  ranks  die  value  of  the  hidden  neurons  in 

ascending  order. 

|  void  rank_hWden  1  _neurons(vokJ) 

r  ( 

bit  1=15; 
bit  number; 
double  numarry[1 6]; 
bitbidxarry[16]; 

'  bitrankanytlG]; 

numarry(0}= 10000;  /*  dummy  entry  7 
for  (number=0;number<15;number++) 

numarryf  number* 1  )=  Hidden1_analog_vaJue[number]; 
lndexx(Lnumarry,bidxarry); 

Rank(l,bidxany,rankarry); 

for  (number^  mjmber<15;number++) 


Hidden  1_ranK_value(numbef]  =  rankarryfnumber+ 1  ]; 


ft************************ *************** 


RANK  HIDDEN2  NEURONS  function  ranks  the  value  of  the  hidden  neurons  In 

ascendfcg  order. 

*********** *«•****••**#***• 

void  ranK.hkfden2L.neurons(void) 

{ 

lntU15; 
int  number, 
double  numanyfl  6); 
int  bKbcany[16t 
int  rankarry(16]; 

numarry(0)>10000;  /*  dummy  entry  7 
for  (numben^number<15;number++) 

numarry(numbef+ 1  }=  Hidden2_analog_value[number]; 

lndexx(i.numarry,fodxarry);  • 

Rankfi.indxarry.rankarry); 

for  (number*0;numbef<1 5;number++) 

Hidden2_ranK_value(number]  =  rankarry[number+1  ]; 

SAVE  CURRENT  WEIGHTS1  function  stores  weight  iayeii  values. 

void  save_current_weights1  (void) 

introw^oi; 

for  (row=0;row<1 6;row++) 
for(col=0;col<  1 6;coi++) 

Tempjayerl  [row][col)=Weight_layer1  [rowflcoi]; 

RESTORE  WEIGHTS1  function  restores  weight  layer!  values, 
void  restore_prtor_weights1(void) 
int  row.col; 

for  (row=0;row<16;row++) 
for(colxO;coi<  1 6;coi++) 

Weigh  t_iayert  [rowJcoi]=Temp_layer  1  [row^coi]; 
if  (OPTICAL*=1)  WRITE_LAYER1_WEIGHTS(); 
else  if  (SOFTWARE_VERSION==2)  Write_sim1_weightsO; 

SAVE  CURRENT  WEIGHTS2  function  stores  weight  Iayer2  values, 
void  8ave_current_weights2(voki) 
int  row.col; 

for  (row=0;row<1 6;row++) 
for(cofc=0;coi<16;coi++) 

Tefnp_layer2(  row](coi}= Weigh  t_layer2[row]Icoi]; 


y«***«** *••*••«••• •»••«•••**••*•*•*«••*•«*•« **•«••****••***»***••»****** 

RESTORE  WEK3HTS2  function  restores  weight  Iayer2  values. 


void  restore_prior_weights2(vokl) 

{ 

fcnt  row, col; 

for  (row*0;row<1 6;row++) 
for(cofcs0;col<16;coi++) 

Weight_layer2[row][coi]=T  emp_layer2[row][col); 
if  (OPTIC ALp»1)  WRiTE_LAYER2_WEIGHTS(); 
else  if  (SOFTWARE_VERSIONs=2)  Write_sJm2_weightsO; 

) 

^SASSMMSSSMS  »***•**••«****«•****•****••#**•**•  *******##0*0 

SAVE  CURRENT  WEIGHTS3  function  stores  weight  Iayer3  values. 

e****************e#*********e*e************«***e******e*******ft*e •*••••**/ 

void  save_current_weights3(void) 

{ 

int  row, col; 

for  (row=0;row<16;row++) 
for(cofc=0;col<3;coi++) 

T ernp_layer3(rowJ[coi)=Weight_iayer3{rowJcol]; 

} 

RESTORE  WEIGHTS3  function  restores  weight  Iayer3  values. 

MMHsesesess«seese»<Hfsssseeesessesstse**»Msess>es*Hessseesssss— sty 

void  restore_pricr_weights3(void) 

{ 

int  row.col; 

for  (row*0;row<1 6;row++) 
for(col=0;col<3;col++) 

Weight_layer3{row][coI]=Temp_layer3{row][cof]; 

} 

ADAPT  WEIGHTS1  function  adapts  one  neuron  at  a  time  until  its  binary  value 
is  inverted. 


void  adapLweights1(int  neuron_number) 

{ 

Int  adapLcounten 
double  adJusLvalue; 
int  row, column; 
int  inverted; 
if  (OPTIC AL=1){ 
compare_hkJdent_neurons(); 
if  (Hidden1_binary_valuefneuron_number]s=:l) 
inverted  a  0; 
else 


Inverted  a  1; 

} 

else  if  (SOFTWARE_VERSION=1 ){ 
generate Jayer1_neurons(); 

R  (Hidden  1_analog_value(neuron_number)=1 ) 
inverted  » -1; 
else 

Inverted  =  1; 


eise{ 

sJmulateJayer1_neuronsO; 

N  (HkJden1_analOQ_value[neuron_number)=1 ) 
inverted  *  0; 
else 

inverted  a  i; 

} 

adjust_value=.005 ; 
gotoxy(1,7); 

printf(‘Hidden1  NeuronAn"); 
printfCValueAn"); 
printf(*BJnary:\n’); 
prlntf("lnvefted:\n"); 

if  ((OPTICAL— =1 )  II  (SOFTWARE_VERSION=2)) 
printf ("Dynamic  Threshold  Layer  1  Output"); 
for  (adapt_counter=0;adapLcounter<1 000;adapLcounter-H-){ 
for  (row  =  0;  row  <  16;  row++)( 


1*  Test  for  adaptation  direction  7 
If  (Hiddenl  Jbinary_value[neuron_nurnber]==1 ){ 
if  (((lnpuLpattem[row]<aO)&&(WeighUayer1  [row][neuron_numberj<0))ll 
((lnput_pattem[row]==1  )&&(WeighUayer1  [row][neuron_number]>0))) 
Welght_layer1(row](neiiron_numberJ  += 
Weight_layerlirow][neuron_number]*(-adiusLyalue); 
else 

WelghL.layer1[row][neuron_number]  += 
Weight_layerl[rowj(neuron_number]‘adjust_value; 

} 

else  { 

if  (((lnput_pattem[row]<=0)&&(WeighUayer1  [row][neuron_number]<0))ll 
((lnput_pattem[row]==1  )&&(WeighUayer1  (row][neuron_number]>0))) 

WeighUayer1Irowjneuron_numberJ  += 

Welght_layer1[rowj[neuron_number]*adjust_vaIue; 

else 

Welght_layer1[row][neuron_number]  += 
WelghUayerlirowjineuron_number]*(-adjusLvalue); 

} 

> 

if  (OPTICALBalK 
WRITE_LAYER1_WEIGHTS(); 
compare _Ndden1_neurons(); 

} 

else  if(SOFTWARE_VERSION=2) 
sknulate.layerljneuronsO; 
else 

genera  te_layer1_neurons0; 

ooloxv(18.7): 

printf("%d",neuron_number); 

dreolO; 

gotoxy(9,8); 

printf("%J2r,Hldden1_analog_value[neuron_number]); 

clreolO; 


136 


gotoxy(10.9); 

prtntf(*%cT(HWden1_bb\ary_valuelneuroo_nurnb©r]); 

dreolQ; 

gotoxy(i2.l0); 
print}(*%d*, Inverted); 
dneolO; 

If  ((OPTIC AL=1)  II  (SOFTWARE_VERSION=2)){ 
gotoxy(35,ll); 

printf(*%^lf*,THRESHOLD1 ); 

dreolO: 

) 

/■Test  if  neuron  is  adapted  7 

if  (Hidden1_binary_value{neuron_number]  ==  inverted)  adapt_counter=1  OOO; 


ADAPT  WEIGHTS2  function  adapts  one  neuron  at  a  time  until  its  binary  value 
Is  inverted. 

ft****#*#*****************************************************************/ 


void  adapt_weights2(int  neuron_number) 

{ 

int  adapLcounter; 
double  adjusLvalue;  * 
int  row.coiumn; 
int  inverted; 
if(OPTICAL=1K 
compare_hkJden2_neurons(); 
if  (HkkJen2_binary_yalue[neuron_number]==1 ) 
inverted  =  -1 ; 
else 

inverted  s  1; 

) 

else  if  (SOFTWARE_VERSION=1 ){ 
generate_layer2_neurons(); 
if  (Hidden2_analog_value[neuron_number]==1 ) 
Inverted  =  -1; 
else 

inverted  =  1 ; 

) 

else{ 

sJmulateJayer2_neuronsO; 

if  (HJdden2_analog_value[neuron_number}=1 ) 
inverted  =  -1; 
else 

inverted  =  1; 

) 

adjust_value=.0O5 ; 
gotoxy(1,7); 

printf(*HJdden2  Neuron:\n*); 
prlntfn/alueAn*); 
printf(*Blnary:\n*); 
printfflnvertedAn*); 

if  ((OPTlCAL=1)  II  (SOFTWARE_VERSION=2)) 


printffDynamic  Threshold  Layer  2  Output*); 
for  (adapt_oounter=0;adapL.counter<1 000;adapL.counter-H-){ 
for  (row  »  0;  row  <  16;  row++){ 
r  Test  for  adaptation  direction  7 
If  (Hklden2_binaiy_vaIue{neuron_number]=1 ){ 
if  (((Hkiden1_blnary_valoe{row]<=0)&&(Weight_layer2(row5neuron_number]<0))ll 
((Hidden  1  _binary_value[row}=*  1  )4&(Weight_iayer2(row|neuron_number]>0))) 
Weight_layer2[rowKneuron_number]  += 
WelQht_layec2[rowfneuron_numberr(*adjusLvalue); 
else 

WeighUayef2[rowXneuron_number]  += 
Weight_layer2[rowXneuron_number]*adjusLvalue; 

) 

else  { 

If  (((Hidden1_blnary_vahie{row]<=0)4A(WelghLlayer2[row][neuron_number3<0))ll 
((Hiddenl  _binary_vakie(rowJ=*1  )&&  (Weigh  t_layer2(row][neuron_number]>0))) 
Wdght_layer2[rowXneuron_number]  += 

Weigh  t_teyer2[row][neuron_numberI*adjusLvalue; 
else 

Wefght_layer2[rowJneuron_number]  += 
WeighLlayer2[rowXneuron_numberr(-adjusLvalue); 

)} 

if  (OPTICAL){ 

WRIT^_LAYER2_WEIGHTS(); 

compare_hidden2_neurons0; 

else  tf(SOFTWARE_VERSION==2) 
simulate_layer2_neurons0; 
else 

generate Jayer2_neurons(); 
gotoxy(17,7); 

printf(*%d*,neuron_number); 

ckeolO; 

gotocy(9.8); 

printf(*%2f‘pHidden2_analog_value[neuron_number]); 

dreolO; 

gotoxy(10,9); 

printf(*%d’,Hidden2_binary_value(neuron_number]); 

dreolO; 

gotoxy(12a10); 

prlntf(*%d*, inverted); 

dreolO; 

If  ((OPTICAU=1)  II  (SOFTWARE_VERSION=2)){ 

Qotoxvf35.il): 

fS(*%.2r,THRESHOLD2); 

dreolO; 

) 

/•Test  If  neuron  is  adapted  7 

if  (Hidden2_blnary_vaiue{neuron_number]  =  inverted)  adapt_counter=1 000; 

)  1 

SINGLET S2  function  adapts  one  neuron  at  a  time.  If  change  results  in 


reducing  Hamming  Distance  then  change  is  kept  Otherwise  weights  are 
restored  to  original  values. 

void  singlets2(int  pattem.number) 

{ 

Int  number. 

int  neuron_number; 

int  adapLnumber; 

unsigned  Int  CurrentHammDist;  /*  Hamming  distance  value  7 
unsigned  int  NewHammDist;  /*  Used  to  compare  hamming  distances  7 
/"Rank  analog  neurons  in  order  closest  to  zero*/ 
rank_hkJden2_neurons{); 

Adapt  in  direction  to  change  corresponding  binary  output 
of  hidden  layer 

for  (adapt_number  *  1;  adapUiumber  <  16;  adapLnumber++){ 
save_current_weights2(); 

CurrentHammDist  =  hamming_cBstance(pattem_number); 
r  Search  for  matching  rank  value  V 

for  (neuron_number=0;  neuron.number  <  IS;  neuron _number++) 
H(Hidden2_rank_vahje{neuron_number]  =  adapt_number){ 
adapt_weights2(neuron_number); 
number^  neuron_number, 
neuron_numben=20;  r  exit  adapt  loop  */ 


generate.neuronsO; 

NewHammDist  =  hamming_distance(pattem_number); 
HammDist=NewHammDist; 

/*  Test  for  adaptation  results  7 
if  (NewHammDist  >=  CurrentHammDist){ 
rastore_prkx_weights20; 
HkJden2_adapted[number]=0; 

generate.neuronsO; 


Hidden2_adapted(nu  mber]=  1 ; 

if  ((NewHammDist  <=  CRITERIA)  &&  (test_output(pattem_number))) 
adapt_number «  20;  TExit  Singlets  7 

1 


DOUBLETS2  function  adapts  two  neurons  at  a  time.  If  change  results  in 
reducing  Hamming  Distance  then  change  is  kept  Otherwise  weights  are 
restored  to  original  values. 

void  doublets2(!nt  pattem_number) 

{ 

int  number!  ,numbet2; 
int  neuronjruimber; 
int  adapLnumber, 

unsigned  int  CurrentHammDist;  t*  Hamming  distance  vakie  7 
unsigned  int  NewHammDist;  /*  Used  to  compare  hamming  distances  7 
/"Rank  analog  neurons  In  order  closest  to  zero7 


rank_hidden2_neoronsO; 

Adapt  in  direction  to  change  corresponding  binary  ou$ut 
of  hidden  layer 

for  (adapLnumber  » 1;  adapLnumber  <  16;  adapt_number++){ 
save_cunenLweights20; 

CurrentHammOist  »  hamming_cHstance(pattem_number); 
r  Search  for  matching  rank  value  and  adapt  in  pairs  neurons  which  have 
not  been  adapted  7 

for  (neuron_number»0;  neuronjuimber  <  IS;  neuron_number-M-) 
*;(Hkiden2_rank_value{neoron_number]  =  adapLnumber) 
&A(Hidden2_adapted[neuron_number]  a  o)H 
adtv>Lweights2(neuron_number); 
numbed  «neuron_number; 
neuron_number*20;  r  exit  adapt  loop*/ 

} 

if  (adapLnumber  <=14) 

for  (neuron_numben=0;  neuron_number  <15;  neuron_mjmber++) 
if((Hidden2_rank_vaIue(neuron_number]  «=  adapLnumber+1) 
&&(Hkfden2jadapted(neuron_number]  =  0)){ 
adapLweights2(neuron_number); 
number2=neuron_numben 
neuron_number=20;  /*  exit  adapt  loop*/ 

} 

generate_neurons(); 

NewHammDist «  hamming_distance(pattem_number); 
HammDist=NewHammDist; 
r  Test  for  adaptation  results  7 
If  (NewHammDist  >=  CurrentHammDist){ 
restore_prior_welghts20 ; 

HkJden2_adapted[number1  }=0; 

HkJden2_adapted(number2}=0; 

genera  te_neuronsO; 

) 

else{ 

HkJden2_adapted(number1]=1 ; 

Hidden2_adapted[numbef2]=1; 

} 

If  ((NewHammDist  <=  CRITERIA)  &&  (test_output(pattern_number))) 
adapt_number  *  20;  /*Exft  Doublets  7 

) 

} 

^ee*****e*e*eee*****ee**e**eeee****e**ee***ee****e*******eae*ee*eftee*e*e*e 

TRIPUETS2  function  adapts  three  neurons  at  a  time.  If  change  results  in 
reducing  Hamming  Distance  then  change  Is  kept  Otherwise  weights  are 
restored  to  original  values. 

•MMSMMSMSMMSMMSSSSMMSMSSSSSftSSMSMSSSMMtMMStftSMMSHSS^ 

void  tripiets2(int  pattem_number) 

{ 

kit  numberl  ,number2,number3; 
kit  neuron_number, 
kit  adapLnumber; 

unsigned  kit  CurrentHammDist;  /*  Hamming  distance  value  7 


unsigned  int  NewHammOist;  /*  Used  to  compare  hamming  dtetances  7 
/•Rank  analog  neurons  In  order  cioeest  to  zero*/ 
rank_hidden2_neuron80; 


Adapt  in  dbedion  to  change  oorrespondng  binary  output 
of  hidden  layer 

for  (adapLnumber «  2;  adapLnumber  <  16;  adapt_number++){ 
save.current_weights20; 

CurrentHammOist  *  hamming_dtetanoe(pattorn_rHsnber); 
r  Search  for  matching  rank  value  and  adapt  In  pairs  neurons  which  have 
not  been  adapted*/ 

for  (neuron_number=0;  neuron_number  <  15;  neuron_number++) 
if((HkJden2_rank_vaiue{neuron_number]  =  adapLnumber) 
&&(Hklden2Ladapted[neunxLnumber]  «  0)H 
adapLweights2(neuron_numbef); 
number!  =neuron_numben 
neuron_number=20;  /*  exit  adapt  loop  V 

if  (adapLnumber  <=14) 

for  (neuron_numben=0;  neuron_number  <  15;  neuron_number++) 
lf{(HJdden2_ranK_value{neuron_number]  =  adapLnumber+1 ) 
&&(Hidden2_adapted(neuron_number]  =  0)){ 
adapLweights2(neuron_number); 
numbei2=neuron_number, 
neuron_number=20;  r  exit  adapt  loop  7 


if  (adapLnumber  <=13) 

for  (neuron_number=0;  neuron_number  <  15;  neuron_number++) 
H((HkJden2_rank_valuelneuron_number]  =  adapLnumber+2) 
&&(Hidden2_adapt8d[neuron_number]  a  0)){ 
adapLweights2(neuron_number); 
number3=neuron_nu  mber; 
neuron_number=20;  /*  exit  adapt  loop  V 

generate.neuronsO; 

NewHammOist  ■  hamming_(fistance(pattem_nurnber); 
HammOisteNewHammOist; 
r  Test  for  adaptation  results  7 
If  (NewHammDbt  >=  CurrentHammDist){ 
iestore_prior_weights20; 

HkJden2_adapted(number1  )=0; 

Hidden2_adapted(numbef2)=0; 

HkJden2_adapted(numbef3}=0; 

generate_neurons(); 

) 

eise{ 

HWden2_adaptefnumber1)=1; 

Hkjden2_adaptet^number2}=1 ; 

HJdden2_adapted[number3]=  1 ; 

) 

if  ((NewHammOist  <=  CRITERIA)  &&  (tesLoutput(pattem_number))) 
adapLnumber  *  20;  /*Exit  Triplets  7 
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QUADLETS2  function  adapts  three  neurons  at  a  time.  If  change  results  in 
reducing  Hamming  Distance  then  change  b  kept  Otherwise  weights  are 
restored  to  original  values. 


void  quadtots2(lnt  pattorn_number) 

hit  number!  tnumber2jiumber3,number4; 
hit  neurotLnumber; 
hit  adapLnumber; 

unsigned  hit  CurrentHammDist;  /*  Hamming  (fistance  value  7 
unsigned  hit  NewHammDist;  /*  Used  to  compare  hamming  distances  7 


/"Rank  analog  neurons  hi  order  closest  to  zero*/ 


ranK_hJdden2_neuronsO; 

Adapt  fodhectkxi  to  change  correspondhig  binary  output 
ofhidden  layer  ^  ^  ^  M 

for  (adapLnumber  « 1;  adapLnumber  <  16;  adapLnumber++){ 
save_curronLweights20; 

CurrentHammDist  *  hamming_dtetance(patt8m_number); 

/*  Search  for  matching  rank  value  and  adapt  in  pahs  neurons  which  have 
not  toen  adapted  7 

for  (neuron_numbereO;  neuron_number  <  15;  neuron_number++) 
if((Hfidden2_raniLvaiue(neuron_nunberi  *=  adapLnumber) 
&&(Hidden2_adaptedP>euron_number]  ■»  0)X 
a<fopLweights2(neuron_number); 
number!  *neuron_number, 
neuron_number=20;  /*  exit  adapt  loop*/ 

} 

if  (adapLnumber  <=14) 

for  (neuroiLnumbefeO;  neuronjiumber  <  15;  neuron_number++) 
if((Hidden2_rank_value(neuron_number]  =*  adapLnumber+1) 
&&(HkJden2_adaptedIneuron_number]  —  0))( 
adapLweights2(neuron_number); 
number2sneuron_number; 


neuron_number=20;  /*  exit  adapt  loop*/ 

1 

if  (adapLnumber  <=13) 

for  (neuron_numbeM);  neuron_number  <  15;  neoron_number++) 
lf((HkJden2_rank_value{neuron_number]  -»  adapLnumber+2) 
&&(HkJden2_adapted(neuron_number]  =  0)K 
adapLweights2(neuron_number); 
number3>neuron_number; 
neuron_number*20;  /*  exit  adapt  loop  7 
) 

if  (adapLnumber  <*12) 

for  (neurorLnumbereO;  neuron_number  <  15;  neuron_number++) 
h((Hidden2_rank_value(neuroa_number) »  adapLnumber+3) 
&&(Hdden2_adapted(neuron_number]  0)H 


) 


■Amt*  iwlnhKtMimiWMi  nmwharV 

number4«neuron_number; 
neuron_number«20;  /*  exit  adapt  loop  7 
} 

generate.neuronsO; 

NewHammOist «  hamrning_dtetance{patt8rn _number); 
HammDtsUNawHarnrnDlst; 
r  Test  for  adaptation  results  V 
If  (NewHammOist  >■  CurrentHammOistK 
restora_prkx_weight820; 

Hldcten^adapted(numb0rt)=O; 

Wdden2_adapted(numbej2H); 

Hidden?_adapt8d(number3Hl; 

Hidden2_adBpted(number4}>0; 

generate.neuronsO; 

} 

abe{ 

HkJden2_adapted(nufnber1)»1; 

Hldd8n2_adapted(number2}si ; 

Hldden2_adapted(number3]si ; 

Hidden2_adapted[numb9r4>>  1 ; 

) 

if  ((NewHammOist  <*  CRITERIA)  &&  (test_output(pattem_number))) 
adapLnumber  ■  20;  TExtt  Quadtets  7 

) 


•**«+«****••* 


SINQLETS1  function  adapts  one  neuron  at  a  time.  If  change  results  in 
reducing  Hamming  Distance  then  change  Is  kept  Otherwise  weights  are 
restored  to  original  values. 

void  singietsl  (kit  pattem_number) 

kit  number; 
kit  neuron_numben 
kit  adapt_number; 

unsigned  kit  CurrentHammDist;  /*  Hamming  cBstanoe  value  7 
unsigned  kit  NewHammOist;/*  Used  to  compare  hamming  dtetances*/ 
/"Rank  analog  neurons  in  order  closest  to  zero*/ 
ranK_hidden1_jieuronsO; 

Adapt  in  dkection  to  change  corresponding  binary  output 
of  hidden  layer 


e**eaae*e*a**e*e*«e*ee*eeee*a*«e 
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for  (adapt_number  »  1;  adapt_number  <  16;  adapLnumber++){ 
save_current_weighta1 0; 

CurrentHammOist  *  hamming.cflstance(pattem_nurnber); 
r  Search  for  matching  rank  value  V 

for  (neuron_number*0;  neuron_number  <  15;  neuron_number++) 
i^tidden1_renk_value{neuron_number] *»  adapLnumberH 
adapLweightsI  (neuron_number); 
numbenmeuron_rMknber; 
neuron_number>20;  r  exit  adapt  loop  7 
} 


generatauneuronsO; 

NiwHftnin<)tet«hiiT«nina  numbsrt*. 


r  Test  lor  Mtaptation  results  */ 
i  (NswHammDIat  >■  CurrentHammOistX 
restore_prior_wsighta  1 0; 
HMden1_«dap*sd(rxjmberH>; 
Mnwn  neurons/): 


HkJdenl  adaptad(number]»1 ; 

I  ((NewHanvnDtet  <*  CRITERIA)  &&  (tesLoutput(pettern_number))) 
adapLnumber  >  20; /*Exit  Singlets  7 


DOUBLETS1  function  adapts  two  neurons  at  a  time.  If  change  results  In 
reducing  Hamming  Distance  then  change  Is  leapt  Otherwise  weights  are 
restored  to  original  values. 

awaa aaaeeeawaeeeeeaae  mihimhs  eeaaewaeeeaaaeeeaeeaeaaawa eaaaawaeaaaaaaa j 


void  doublets1(int  pattsmjiumber) 
{ 

int  number!  ,number2; 


bit  neuron-number; 
bit  adapLnumber; 

unsigned  bit  CurrentHammOist;  /*  Hamming  distance  value  V 
unsigned  bit  NewHammOist;  /*  Used  to  compare  hamming  dtetances  7 
/*Rank  analog  neurons  in  order  closest  to  zero*/ 
rank_hlddsn1  jeurorwQ; 

awfb^^twswt^^^^b^^^SrtMbSMMiw^fMbfMbwwrbsiwwrtaeaaaaaeaaeaaeaae  aaawaaaaaeaaaaaaaaaaaa 


Adapt  bi  dbectkm  to  change  oorrespondbig  bbiary  output 
of  hidden  layer 


7 


for  (adapLnumber  « 1;  iKiapLnumber  <  16;  adapLnumber++){ 

save_currenLwelghts1 0; 

CurrentHammOist *  hamming_dtetanco(pattem_number); 

/*  Search  for  matching  rank  value  and  adapt  bi  pairs  neurons  which  have 

not  been  adapted*/ 

for  (neuroruMJmben-O;  neuron_number  <  15;  neuron_number++) 
lf((HWden1_rank_vaJue{r>euron_number]  =  adapLnumber) 

&&(HJdden  1  _adapted[neuron_number]  =»  0)K 
adapLweights  1  (neuron_number); 
numberi=neuron_number; 
neuron_number*20;  /“exit  adapt  loop*/ 

) 

if  (adapLnumber  o14) 

for  (neuron_numbert);  neuroruiumber  <  15;  neoron_nomber++) 
if((Hkjden  1  _rank_vakie(neuron_number]  a  adapLnumber+1) 
&&(Wdden1_adapted(neuron_number]  =  0)){ 
adapLweightsI  (neuron_number); 
number2*neuron_number; 
neuron_number=20;  /*  exit  adapt  loop  7 
) 

generate jneuronsO; 

NewHammOist »  hammbig_(8stance(pattem_nombef); 


HammDisUNewHammDist; 
r  Test  for  adaptation  results  7 
if  (NewHammDtet  >-  CurrsntHammDistH 
restore jjftor.wsightsl  0; 

Hidden1_adapted(mimber!}«0; 

Htoden1j&taptod(numfeier2)=0; 

generate_neurons{); 

} 

eise{ 

Hidden1jadapted(number1}=1; 

Hldden1_adapted(nuTnber;=1 ; 

} 

if  ((NewHammDist  <■  CRITERIA)  &&  (test_output(petiem_number))) 
adapLnumber  *  20;  r  Exit  Doublets  7 

1 

) 

TRIPLETS1  function  adapts  three  neurons  at  a  time.  If  change  results  in 
reducing  Hamming  Distance  then  change  is  kept  Otherwise  weights  are 
restored  to  original  values. 

void  tripietsl  (int  pattem.number) 

{ 

int  number!  ,number2,number3; 
int  neuron_numben 
bit  adapLnumber; 

unsigned  Int  CurrentHammDist;/*  Hamming  distance  value  */ 
unsigned  bit  NewHammDist;  /*  Used  to  compare  hamming  (Sstances  7 
/•Rank  analog  neurons  bi  order  closest  to  zero*/ 
rank_hldden  1  _neurons(); 

Adapt  bi  direction  to  change  corresponding  binary  output 
of  hidden  layer 

for  (adapLnumber  =  1;  adapLnumber  <  16;  adapLnumber++){ 
savejcurrenLweightsI  0; 

CurrentHammDist  =  hamming_distance(pattem_number); 

/*  Search  for  matching  rank  value  and  adapt  in  pairs  neurons  which  have 
not  been  adapted*/ 

for  (neuron_number=0;  neuron.number  <  15;  neuron_number-H-) 
if((Hklden  1  _rank_value[neuron_number]  =  adapLnumber) 
&&(Hidden1_adapted[neuron_number]  =  0)H 
adapLweightsI  (neuron.number); 
numberl  =neuron_numben 
neuron_number=20;  /*  exit  adapt  loop*/ 

if  (adapLnumber  <=14) 

for  (neuronjfiumber=0;  neuronjiumber  <15;  neuron_number++) 
if  ((Hidden  1  _rank_value(neuron_number]  =  adapLnumber+1) 
&&(HkJden1_adapted(neuron_number]  =  0)){ 
adapLweightsI  (neuron.number); 
number2=neuron_number; 
neuroo_number=20;  /*  exit  adapt  loop  7 


N  (adapUwmber  <*13) 

for  (neuron_numben=0;  neuron_number  <  15;  neuron_number++) 
H((Hidden  1  _ranK_value(r>euron_numberl  ■»  adap(_number+2) 
&&(Hidden  1  _adepted(neuron_number]  ■»  0)){ 
adaptjweightsl  (neuron_number); 
number3«n©uron_number; 
neuron_number*20;  /*  exit  adapt  loop  V 
} 

generate_neuronsO; 

NewHammDist  *  hamming_distance{pattern_number); 
HammDisUNewHammDist; 
r  Test  for  adaptation  results  7 
if  (NewHammOist  >»  CurrentHammDist){ 
restore jprior jweightsl  0; 

Hidden  1_a<taptedfnumber1  ]=0; 

HkJden1_«japted(number:*0; 

HJdden1_adapted(number3}=0; 

generate.neuronsO; 

} 

etse{ 

Hidden1_adapted(number1  >=1 ; 

HkJden1_adaptedlnumber2)=»1 ; 

Hidden  1  _adapted(number3}=  1 ; 

} 

if  ((NewHammDist  <=  CRITERIA)  &&  (test_output(pattern_number))) 
adapt_number  =  20;  TExtt  Triplets  7 

} 

^••i*Ms**tt***Mft*«**«*s*s*esft*ss****tftt«ftt*ss**ss**ts«H«i*s«s*ts**e**ft 

QUAOLETS1  function  adapts  three  neurons  at  a  time,  if  change  results  in 
reducing  Hamming  Distance  then  change  is  kept  Otherwise  weights  are 
restored  to  original  values. 


void  quadtets1(lnt  pattem_number) 

{ 

hit  numberl  ,number2,number3,number4; 
bit  neuron_number; 
bitadapCnumber; 

unsigned  bit  CurrentHammDist;  /*  Hamming  distance  value  7 
unsigned  bit  NewHammDist;  I*  Used  to  compare  hamming  distances  7 

/•Rank  analog  neurons  hi  order  closest  to  zero*/ 

ranK.hkfden1_neurons0;  ^ 

Adapt  bi  direction  to  change  corresponding  binary  output 
of  hidden  layer 

for  (adapLnumber  *  1;  adapLnumber  <  16;  adapLnumber++){ 
savejcurrenLweightsI  0: 

CurrentHammDist  =  hamming_distance<pattem_number); 

/*  Search  tor  matchbig  rank  value  and  adapt  bi  pairs  neurons  which  have 
not  been  adapted  7 
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for  (neufoa.number*0;  neuronjnumber  <  IS;  neuron_number++) 

lf((HkJden1  _ranK_value(n©uron_number] as  adapLnumber) 
&A(HMden1_adapted[neuron_number)  as  o)K 
adapLweightsI  (neuron_number); 
number!  *neuron_number; 
neuron_number=20;  /*  exit  adapt  loop*/ 

1 

If  (adapLnumber  <=14) 

for  (neuron_numbef=0;  neuron_number  <  15;  neuron_number++) 
lf{(Hldden1_ranK_vaJue{neuron_number]  as  adapLnumber+1) 
&&(Hklden1_adapted(neuron_number]  =  0)){ 
adapLweightsI  (neuron_numb or); 
number2aneuron_number, 

neuron_number=20*,  /*  exit  adapt  loop*/ 

) 

If  (adapLnumber  <=13) 

for  (neuron_number=0;  neuron_number  <15;  neuron_nurnber++) 
lf((HkJden1_rank_value(netiron_number]  =  adapLnumber+2) 
&&(Hidden1_adapted(neuron_number]  =  0)){ 
adapLweightsI  (neuron_number); 
number3*neuron_numben 
r>euron_number=20;  /*  exit  adapt  loop  7 
} 

if  (adapt_number  <=12) 

for  (neuron_number*0;  neuron_number  <15;  neuron_number++) 
if  ((Hidden  1  _ranK_value(neuron_number]  as  adapLnumber+3) 
&&  (Hidden  1  _adapted(neuron_number]  at  0)){ 
adapLweightsI  (neuron_number); 
number4sneuron_number; 
neuron_number=20;  /*  exit  adapt  loop*/ 

) 

generate.neuronsO; 

NewHammDist  a  hamming_dlstanc8(pattem_nunr>ber); 

HammDist=NewHammDist; 

/*  Test  for  adaptation  results  7 

if  (NewHammDist  >=  CurrentHammDist){ 
restore_prior_welghts1 0; 

HkJden1_adapte#uimber1]sO; 

Hidden  1_adapted{number2>=0; 

Hiddenl  _adapted(number3]=0; 

H(dden1_adapted{number4}=0; 

generate_neuronsO; 

1 

else{ 

HkJden1_adapted(number1}=1; 

Hidden1_adapted[number2)ai ; 

Hldden1_adapted[number3)=1 ; 

Hidden1_adapted[number4]=1 ; 

} 

If  ((NewHammDist  <=  CRITERIA)  &&  (tesLouiput(pattem_number))) 
adapLnumber  a  20;  /*Exit  Quadtets  7 


adapt  WBQHTS  3  junction  adapts  third  layer  neurons. 

void  adapt_weights3(int  neuron_number) 

IntadapLcountar, 
double  adjusLvatue; 

Int  row, column; 
int  inverted; 
float  threshold; 

K  (OutpuLanalogjvalue{neuron_numberl>aO) 

Inverted  a  O; 
else 

Inverted  =  1; 
adjust_value«.005 ; 

Ootoxy(1,13); 
printfCOuiput  neuronAn*); 
prlntfCValueAn*); 
p^ntf('Blnaiy:\n*); 
printf('lnverted:'); 

lor  (adapLcounter=0;adapt_counter<1 000;adapt_counter++){ 
for  (row  c  0;  row  <  16;  row++){ 

/*  Test  for  adaptation  (fraction  7 
If  (CXitputJt)inary_value{neuron_number)=  1 ){ 

If  (((HJdden2_blnary_value[row]<0);i&(WeighLlayer3[row][neuron_number]<0))ll 
((Hkfden2_binary_value{row)>0)&&(WeightJayer3[row][neuron_nurnber]>0))) 
WeightJaye^row][neuron.numt)er]  += 
Welght_layer3(rowJneuron_numberr(-adjust_value); 
else 

Woighf_layer3{rowjneuron_number]  += 
Welght_layer3lrow5neuron_numberj*adJusLvalue; 

else  { 

if^l!^^“nary-valueIrowl<0)&&(Wei9hLlayer3(row][neuron_number]<0))ll 
((Nfdden2_binary_vakie{row}>0)&&(WeighLlayer3{rowKr>euron_number]>0))) 
Welght_layer3(rowJneuron_number]  += 
WeighLlayer3lrowJneuron_number]*adjusLvalue; 
else 

Welght_layer3{rowJneuron_number)  += 
Wefght_layer3trow5neuron_number]*(-adJust_value); 

} 

generate_output.neuronsO; 

gotoxy(15,l3); 

prtatf(*%d*,neuronjnumber); 

dreoK); 

flotoxy(9,14); 

printf(*%^p,Ouiput_anatogLvalue[neuron_numberJ); 

dreoK); 

QOtoxy<10,15); 

pr1ntf(*%d*  ,OutpuLbinary_vaiue{neuron_number]); 

dreoK); 

gotoxy(12,16); 

printfC%<r,lnverted); 

dreoK); 

rTest  if  neuron  is  adapted  V 


If  (Output_Wnary_value(neuron_number]  =  inverted)  adapLcounter= 1 000; 

} 

} 

yeeeee*e*ee#e*e**eeee*ee**#e**ee*ewe*e**e*ee*e ********** eeee ••***«••# ***** 

ADAPT  OUTPUT  LAYER  function  adapts  one  neuron  at  a  time  untH  its  binary 
value  is  Inverted. 

a********************************* ********** a************ ************* ***^ 

void  adapLoutpuUayer(int  pattem_number) 

( 

unsigned  bit  actualjvector, 
unsigned  bit  ex_or_vector; 
unsigned  bit  adapt(3]; 
bit  number; 
bit  adapLnumber, 
bit  column; 

generate_output_neurons(); 

save_currentlwelghts3(); 

generate_output_neuronsO; 

actuaLvector  » (Output_binai  y_value[0]  *  1)  +  (OutpuLbinary_value[1  ]  *  2)  + 
(OutpuLbbiary_value{2]  *  4); 

PX_or_vector »  actuaLvector  A  Letter Jarget(pattem_number]; 

r************************************************************************ 

Adapt  bi  direction  to  match  corresponding  binary  output 

aM*s**a*Mae*M****«Maww« «•**•****•**•*«*****• ******************e*******^ 

for  (number=0;number<3;number++){ 

adapt(number)=  (ex_or_vector  &  (unsigned  bit)  pow(2,number))»number, 
if  (adapt(numbe?;a1 ) 

adapt_weights3(number); 

generate_outpuLneurons(); 

HammDist  *  hamming_distanc8(pattem_number) ; 
if  (HammDist  <=CRITERIA) 

prbitffConresponding  output  neurons  adapted,  get  next  pattern  \n*); 
else  { 

restore_priorjweights30; 

generate_output_neurons(); 

} 

) 

/**•******•***•**“****“***•*****•**********•**—***•••**•****“*—*•**** 

FILE:  NETWORK3.C 
WRITTEN  BY:  William  Robinson 

DESCRIPTION:  Saves  and  recalls  weight  values  from  a  user  specified  file. 

r**‘***M***“*****‘**‘******************“*******“—**“*‘*—********** 

SAVE  WEIGHTS  LAYERS  saves  WekjhtJayerl  and  WeighUaywZ  arrays  to  a  file. 

bit  savejNeighUayers(void) 

( 

bit  raw, column; 

if  ((ouLffle  s  fopen(ouLname,  V))  =  NULL) 

{ 

drscrQ; 

fprintf(stderr,  "Cannot  open  output  fite.Nn"); 


for  (row=0;row<16;row++) 
for  (column>0;cofomn<16;column++) 
fprlntf(out_fiJe,  ‘%An',W0ight_layerl  [rowjcolumn]); 
for  (row=0;row<1 6;row++) 
for  (coJumn==0;cohjmn<16;column-H-) 
fprintf(out_file.  "%fta',WelghUayer2[row]{column]); 
for  (row=0;row<3;row++) 
for  (column=0;column<1 6;column++) 
fprintf(out_file,  *%ftn',Welght_layer3[row][column]); 
for  (row=0;row<24;row++) 

fprintf(out_f*te.  ■Pattem{%d]=%d\n’,row,pattem_leamed[row]); 
fprintf(out_fite,  'Iterations  performed:=%<hn\  Iterations) ; 

/*Ctose  the  file*/ 
fcJose(out_file); 
return  0*. 

} 

r************************************************************************** 

GET  WEIGHT  LAYERS  retrieves  weight  layers  1  and  2  from  a  file  and  assigns 
them  to  WelghUayerl  and  WeighUayer2  arrays. 

int  get_weighUayers(vokJ) 

{ 

int  row, column; 
double  temp; 

if  ((injile  =  fopen(in_oame,  V))=  NULL) 

{ 

drscrO; 

fprintf(stderr, ' Cannot  open  Input  file.Vi'); 
return  1; 

} 

for  (row=0;row<1 6;row++) 
for  (column=0;column<1 6;column++){ 
fscanf(in_flJe,  ”%lf\n*,&temp); 

Weight_layer1Irow][column]=temp; 

} 

for  (row=0;row<1 6;row++) 
for  (column=0;column<1 6;column++){ 
fscanffin Jile,  *%lf\n',&temp); 

Weight_layer2[row][column]=temp; 

) 

for  (row=0;row<3;row-H-) 
for  (column=0;column<1 6;column++){ 
fscanf(in_flle,  *%lf\n\&temp); 

WelghUayer3{row][coiumn]=temp; 

} 

/•Close  the  file*/ 
fdose(lnjile); 
return  0; 

} 

yM************************************************************************ 

FILE :  SIM3_D.C 
WRITTEN  BY;  William  Robinson 


DESCRIPTION;  Simulates  Optical  ANN  hardware  and  trains  with  MR2  Rule. 
Incorporates  input  dependent  dynamic  thresholding. 


A*************************************************************** 


DYNAMIC  THRESHOLD  function  calculates  the  threshold  sum  for  each  Input 
pattern  through  column  sixteen. 


a*************** ••**«*** 


double  dynamic  Jhreshokl(int  inputyl  6]) 

{ 

bit  input_neuron; 
bit  OpticaLmld  =  90; 
float  sum; 
sum=0; 

for  (biput_neuron=0;inpuL.neuron<16;input_neuron-H-) 
sum+3  (input[lnput_neuron]*(OpticaLmid)); 
retum(sum); 

} 

y************ ************* ft************************************************** 

WRITE  SIM1  WEIGHTS  function  converts  the  weights  and  biases  into  usable 
forms  for  the  optical  simulation  weights 

void  Write.siml  weights(void) 

{ 

bit  l,J; 

for  (I  s  0;  I  <  15;  I++) 
for  (|  s  0;  J  <  15;  j++)  { 
if  (Welghtjayerip][j]  <  -€0) 

SimJayeriPHfl  =  30; 
else  H(WelghtJayerip][fl  >  60) 

SimJayeripjQ]  =  150; 
else 

Slm_layerip][j]  =  pnt)WeighLlayer1  PKD+90; 

} 

) 

^**************************************************************************** 

WRITE  SIM2  WEIGHTS  function  converts  the  weights  and  biases  into  usable 
forms  for  the  optical  simulation  weights 

void  Wrfte_sim2_welghts(vold) 

{ 

bit  i,J; 

for  p  s  0;  I  <  15;  k+) 
for(|s0;j<l5;l++){ 
if  (Weight_layer2[n01  <  *60) 

Sim Jayer2P]Q]  =  30; 
else  M(WeighLlayer2p](D  >  60) 

Sbn_layer2PK0  =  150; 
dlso 

Sim_layer2P]0]  ■  pnt)WeighUayer2p][jl+90; 

) 

) 

y******* ****************** A*********************************************** 

SIMULTATE  LAYER  1  NEURONS  function  calculates  the  weighted  sum  for  each 
neuron  In  layer  1. 

void  simulate  Jayer1„neurons(void) 

{ 

bit  number; 
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int  row.cotumn; 

Wrlte_sJm1_we)ghtsO; 

THRESHOLD  Is  (dynamic_threshotd(lnput_pattem)); 


*  Generate  Hidden  Layer  1  Neurons 


•••*****•••« a************ aaaaaaaaaaaaaa/ 


/•initialize  values  7 


for  (number=0;number<16;number++) 

Hickjen1_anaJog_value[number)=b; 
for  (number^number<16;number++) 
for  (cx)iumn=0;coiumn<1 6;coiumn++) 

Hidden  1_analog_vaiue[number]  +=  lnput_pattem[column]  *  Sim Jayerl  [coiumn][number]; 
for  (number=0;number<1 5;number++){ 

if  (Hidden  1_anatog_value[number]  >=  (dynamic_threshokf(lnpuLpattem») 
Hidden1_binary_value{number]  =  1; 
else 


HkJden1_binary_value[number]  =  0; 

} 

Hidden  1  _binary_value[1 5)  =  1;  /*Bia$  neuron  7 
} _ 

SIMULATE  LAYER  2  NEURONS  function  calculates  the  weighted  sum  for  each 
neuron  in  layer  2. 

aaaaaaaaaaaaaaaaaaaaaeaaaaaaaaaaaaaaaaeaaeaaaeaaaaaaeaaaaeeeaeaeeaaaaeeea/ 


void  simulate  Jayer2_neurons(void) 

( 

int  number; , 
int  row.coiumn; 

Write_sim2_weights0; 

THRESHOLDS  (dynamic_threshoid(Hidden1_binary_value)); 


/“"Generate  Hidden  Layer  2  Neurons 


aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/ 


/•initialize  values  */ 


for  (number=0;number<16;number-H-) 

HkJden2_anaJog_value[number)=0; 
for  (number=0;number<1 6;number++) 
for  (cdumn=Q;column<1 6;coiumn++) 

Hidden2_analog_vaiue[number]  += 

Hidden  1_binary_value[coiumn]  *  Sim_layer2[coiumn][number]; 
for  (numben=0;number<1 5;number++){ 

H(Hidden2_analog_value(number]>=(dynamicJhreshold(Hidden1_binary_value))) 

Hidden2_binary_value(number]  a  1; 
else 


HkJden2_binary_value[number]  a  -1; 

} 

Hidden2_binary_value[1 5]  a  1;  I*  Bias  neuron  */ 
} 

yf*eaeeaaa**eAaaaaeae*eeeeee***eeeeeeaaeeeeeeeeeeaeeeee  i 


Madaiine  Rule  Two  Algorithm 

•a***************************#*******************************#**************. 


void  simulate(void) 

{ 

int  outer Joop; 

int  pattem_counten  /*  pattern  number  V 

inti; 

clrscrO; 

gotoxyO.2); 

printf(TRA!NING...\n*); 


printffHamming  Distance  Criteria  is:  %d*  .CRITERIA); 
gotoxy(1,4); 

printff Adapting  Weigths  for  pattern:  *); 

/*  Uniform  dtetributkxi  of  weights  7 
initialize.weightsO; 

/>****BOQln  ^lypj^HHeeeeitHeHeHiHssHseeHHeeieseHSisstHHet 

for  (outer Joop^;outerJoop<lteratk)ns;outerJoo{>f+){ 
for  (pattem_oountef=0;pattem_coonter<24j>attefn_counter++){ 
/*pattem_countet=2;  7 
Initlanze.adaptedO; 


'Present  Input  vector** 


/ 


bJnarize_inputs(Letter_bb(pattem_counter],  Letter_msb[pattem_counter]); 


/* . Check  output  vector  and  calculate  Hamming  Distance***************/ 

simulateJayerl.neuronsO: 
simulate Jayer2_neurons0: 
generate_output_neuronsO: 
gotaxy(31,4); 

dreolO; 

gotoxy(31,4); 

print}(*%d*,pattem_counter); 

HammDist  =  hammlng_distance(pattem_counter); 

U  (HammDist  >  CRITERIA) 
slngletsl  (pattem_oounter); 
if  (HammDist  >  CRITERIA) 
doubletsl  (pattem_counter); 
if  (HammDist  >  CRITERIA) 
tripletsl  (pattem_counter); 
if  (HammDist  >  CRITERIA) 
quadletsl  (pattem_oounter); 
if  (HammDist  >  CRITERIA) 
singlets2(pattem_counter); 

If  (HammDist  >  CRITERIA) 
doublets2(pattern_counter); 
if  (HammDist  >  CRITERIA) 
triplets2(pattem_counter); 
if  (HammDist  >  CRITERIA) 
quadlets2(pattsm_counter); 
if  (HammDist  >  CRITERIA) 
adapLoutpuL>ayer(pattem_counter); 
if  (HammDist  >  CRITERIA) 
pattem_leamed(pattem_counter]=0; 
else 


pattern Jeame<flpattem_cou  nter)=  1 ; 


Apply  next  pattern 

************************ 

) 

) 

Store  weights  in  network  We  and  exit 

7 


save_weighUayersO; 

I 
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FILE:  0PTJCAL3.C 
WRITTEN  BY:  WMam  Robinson 

DESCRIPTION:  Performs  Madeline  Rule  2  training  wtih  optical  hardware. 


READ_HIDOEN1_BINARY  function  reads  binary  hidden  outputs  and  assign  to 
neurons  in  hidden  layer .  Function  is  used  for  testing  binary  outputs 
generated  by  the  hardware  and  is  not  used  during  training. 


void  readLhidden1_binary(void) 

l 

unsigned  char  hkfdenl.vectorjow; 
unsigned  char  hidden  1  _vector_h Jgh; 
int  number; 

hidden  1  _vector_tow=4nportb(HOUT_LO) ; 
hidden1_vector_highsinportb(HOUT_Hi): 
for  (numbensO;  number  <8;  number++){ 

Hiddenl  _binaiy_vaiue{number]  = 

(hidden  1  _vector_low  &  (unsigned  int)  pow(2.number))»number; 
Hidden  1  _binary_value{number+8)= 

(hidden  1  _vector_h)gh  &  (unsigned  int)  pow(2,number))»number; 

) 


Hiddenl  _binaiy_vakie(1 5] » 1;  /*BIAS  INPUT  TO  LAYER  2*/ 

) 


************ 


READ_HIDDEN2_BINARY  function  reads  binary  hidden  outputs  and  assign  to 
neurons  in  hidden  layer.  Function  is  used  for  testing  binary  outputs 
generated  by  the  hardware  and  is  not  used  during  training. 

void  read_hJdden2_binary(void) 

{ 

unsigned  char  hidden2_vector Jow; 
unsigned  char  hkJden2_vector_high; 
int  number; 

hidden2_vector_low=inportb(HOUT_LO) ; 
hidden2_vector_highxJnportb(HOUT_HI); 
for  (number=0;  number <8;  number-M-){ 

Hidden2_binaiy_value(number]  = 

(hidden2_vectorjow  &  (unsigned  int)  pow(2,number))»number; 
HJdden2_binary_vatue[number+8]= 

(hkfden2_.vector.high  &  (unsigned  int)  pow(2,number))»number; 

) 

r  Feeds -1  or  +1  to  software  layer  3  which  accepts  symmetric  +/-1  inputs*/ 
for  (number=0;  number<15;  number++K 
if  (IHidden2_binary_value{number]) 

Hldden2_binary_value(number]  =  *1; 

Hk«en2_binaiy_vaiue{151  =  1;  f*  BIAS  INPUT  TO  LAYER  3V 

lmmmuiiimu 

READ  HiDDENI  ANALOG  reads  optical  layer  1  output  neurons  analog  value, 
void  read_Mdden1_analog(void) 
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READ  HID0EN2  ANALOG  reads  optical  layer  1  output  neurons  analog  value. 


void  reaOidden2Lanalog(void) 

{ 

ADCJ-AYER20; 

} 


COMPARE  HIDDEN1  NEURONS  function  reads  ADC  layer  1  channels  and  compares 
their  value  against  threshold  column  for  binary  output  determination. 


void  compareJiidden1_neuroris(void) 

{ 

Int  neuron; 

read  hiddenl  anaiooO: 

THRESHOLD?  aBKcfen  1  _anatog_value{1 5]; 
for  (neuron*0;neuron<1 5;neuron++){ 

If  (Hidden  1  _anak>g_value{neuronj  >=  HMden1_anak>g_value{15]) 
Hldden1_binary_value[neuron]=1 ; 
else 


Hiddenl  j)kiary_vafue(neuron]=0; 

HkWen1_binary_value[1 5]  a  1;  TBlas  neuron  7 

COMPARE  HIDDEN2  NEURONS  function  reads  ADC  layer  1  channels  and  compares 
their  value  against  threshold  column  for  binary  output  determination. 

IMMSMSMSMSSSSSMMSIMMtMSSSMMStSMStMSSSSSSSSMSMMSMSMMS*****^ 

void  compareJiidden2_neurons(void) 

{ 

kit  neuron; 

read_hidden2_analog0; 

THRESHOLD2=Hidden2_analog_value[15]; 
for  (neuron=0;neuron<1 5;neuron++){ 

if  (Hidden2_analog_value{neuron]  >=  HkJden2_analog_vaIue[1 5]) 
HWden2_bkiary_vahje{neuron}=  1  ; 
else 


HkJden2_binary_value[neuron}=- 1 ; 

} 

Hidden2J>kiary_vafue[15]  =  1;  /*Bias  neuron  V 


OPTICAL  ANN  function  performs  Madalkie  Two  Rule  training  on  Optical 
ANN  system. 


void  opUcal_ann(voJd) 

{ 

bit  outer  Joop; 

kit  pattem_counter  /*  pattern  number  V 
inti; 

MODE_OQ;  f  Configure  interface  board  Inside  386  computer  V 

ENABLE_WEIGHTSO;  /*  Allows  SLM  to  be  programmed  V 


dracrO; 

gotoxyCI.2); 

printffTRAlNING...\n*); 

printf (Hamming  Distance  Criteria  is:  %d*  .CRITERIA); 
gotoxWl,4); 

printf(' Adapting  Weigths  for  pattern:  *); 
r  Uniform  dstributton  of  anights  7 
InitlaHzejweightsO; 

Get  weight  layers  1  and  2  stored  in  computer  memory  and  transfer 
to  Opted  hardware  weights  using  interface  board. 

WRITE_LAYER1_WEIGHTS0; 

WRITE_LAYER2LWEIGHTS0; 

/*****Begin  fra|n|ng,*****^****“****M*****#****“*******^**M"M*M/ 
for  (outer_loop^);outer_loop<iterations;outer_kx)p^){ 
for  (pattem_oounter^-.pattem_counter<24  pattem_counler++){ 
/*pattem_counter*2;  7 
initialize.adaptedO; 

/*******  Present  Input  WC|or,w-wwww,t,MM,“,w“‘H‘,,‘ . / 

binarize Jnputs(l*tterJsb(patlem_coun  ter], 
Letter_msb(pettem_counterI); 

VVRITE.INPUT_VECTOR(LetterJsb(pattem_countar]. 

Letter_msb(pattem_counterD ; 
compare_hkfden1_neurons0; 
PRESENT_HIDDEN1_VECTOR0; 
compare_hldden2_neurons0; 
gotacy(31,4); 

dreoK): 

gotaxy(31.4); 

printf(*%d*,pattem_counter); 

/*******Check  output  vector  and  calculate  Hamming  Distance******* * 
HammDist  -  hamming_cS3tanc8<pattem_counter); 
if  (HammDist  >  CRITERIA) 
singietsl  (pattem.counter); 
if  (HammDist  >  CRITERIA) 
doubietsl  (pattem.counter); 
if  (HammDist  >  CRITERIA) 
tripietsl  (pattem.counter); 
if  (HammDist  >  CRITERIA) 
quadtotsl  (pattem.counter); 
if  (HammDist  >  CRITERIA) 

•  singiets2  (pattem.counter); 

If  (HammDist  >  CRITERIA) 
doubiets2(pettem_counter); 
if  (HammDist  >  CRITERIA) 
tripists2(pattem_counter); 
if  (HammOist  >  CRITERIA) 
quadtots2(pattem_counter); 

If  (HammDist  >  CRITERIA) 
adapLoutpuLlayer(pattem_counter); 

If  (HammDist  >  CRITERIA) 
patterrUeemed[pattern_counter}=0; 
else 

pattern Jeamed(pattem_counter]=  1 ; 


Apply  next  pattern 


*****************/ 


Store  weights  in  network  fle  and  exit 


********* A***************** ******** / 


savejweighUayersQ; 


^************** ******************* ***************»>*********•* ************ 

RLE :  3LAYER.C 
WRITTEN  BY:  WMam  Robinson 

DESCRIPTION:  T rains  a  three  layer  ANN  consisting  of  16x1-16x1-5x1 

with  MR2  Rule.  Provides  user  with  comparison  against  simulation  and 
optical  hardware  performance. 

**•♦*•»**•♦*•*—****•**—*———————****——•**—**♦—**•**♦******•/ 


GENERATE  LAYER  1  NEURONS  function  calculates  the  weighted  sum  for  each 
neuron  In  layerl .  ^  ^  ^  ^  ^ 

void  generatejayer1_neurons(void) 

{ 

int  number; 

Int  column; 

/••"Generate  Hidden  Layer  1  Neurons  . . 

/•initialize  values*/ 

for  (number^number<16;number++) 

Hidden  1  _anaiog_value{number)=0; 
for  (number=0;number<1 6;number++) 
for  (coiumn=0;ooiumn<1 6;column++) 

Hidden  1  _anafog_value{number]  += 

InpuLpattemfcolumn]  *  WeighUayerl  [coJumnlnumber]; 
for  (number*^);  numberd 5;number++) 

if  (Hidden  1  _anaiog_value{number]  >=  0) 

Hidden1_binary_value(number]  =  1; 
else 

Hiddenl  Jbbiary.valuefnumber]  =  >1; 

Hidden  1  _binary_vaiue[1 5]  =  1;  /*Bias  neuron  V 

^ _ _ _ _ _ _  _ _ _ 

GENERATE  LAYER  2  NEURONS  function  calculates  the  weighted  sum  for  each 
matron  in  layer  2.  ,«  Ill((l 

void  generate Jayer2jreurons(void) 

{ 

kit  number, 
kit  column; 

/••"Generate  htfdden  Layer  2  Neurons  *•«*•”•**“•'*••••««••••*“•*******/ 

rkiitiatize  values*/ 

for  (number=0;number<1 6;number++) 

Hidden2_anaiog_vakje{number)=0; 
for  (numben*0;number<  1 6;number++) 


for  (column^columfKl6$olumn++) 
Hklden2^analoavaluefnumberi  +« 

Hidden  1  _|3foary_value{cokimn)  *  WeighUayer2(coiumnXnumber}; 
for  (nurnber^number<15;number-H-) 
K(Hidden^anafog_value{numberl  x>  0) 
Wdden2_Wnaiy_valuB{rHimber]  *  1; 


Hidden2jainary_vahie[number] « -1; 
Wdden2_bfoaiy_value{1 5]  *  1;  TBIas  neuron  7 
} 

JWWw  WWWWW^^WWWWW  WWWWWW  WWWWW  W  WWW  W  V  wWW  ■  Vv  VIIIHR 


Madeline  Ride  Two  Algorithm 

iHHeeeeeHeeeeeeeteeeeeeeeeeeeeeeeeee 

void  normaUrainlng(void) 


int  outer Joop; 

kit  pattemjoounter;  /*  pattern  number  */ 

Int  4 

drscrO; 

gotoxy(1j2); 

printf(*TRAlNING...\n*); 

printffHamming  Distance  Criteria  is:  %d*. CRITERIA); 
gotoxy(1 ,4); 

pdntff 1 'Adapting  Weigths  for  pattern: "); 

/*  Uniform  dtetrfoution  of  weights  V 
initialize.weightsO; 

/ - Begin  training***'**************'* . * . 

for  (outerjoop^outerjoop<ltefations;outerjoop^){ 
for  (pattem_oounteraO:pattem_count8r<24^)attem_counter++){ 
Wteltze_adaptedO; 

/ftMMMPfB88nt  Input 

binartzGjnputs(Letter_lsb{pattem_counter]1 

Letter_msb(pattem_counterD; 

/*******Check  output  vector  and  calculate  Hamming  Distance******** 
generate Jayw1_neurons0; 
generate Jayer2_neurons0; 
generate_output_neuron80; 

Qotacy(31,4); 

dreolO: 

flotaxy(31,4); 

printf(“%d*4»ttern_counter); 

HammDist »  hamming_dtetance(pattern_counter); 

If  (HammDist  >  CRITERIA) 
slngletsl  (pattem_counter); 
if  (HammDist  >  CRITERIA) 
doubietsl  (pattem.counter); 
if  (HammDist  >  CRITERIA) 
tripietsl  (pettem_counter); 
if  (HammDist  >  CRITERIA) 
quadetsl  (pattem_counter); 

I  (HammOist  >  CRITERIA) 
singiets2(pattem_counter); 
if  (HammDist  >  CRITERIA) 
doubiets2(pettem_oounter); 


*Ht**MH*tM*  1 


if  (HammDbt  >  CRITERIA) 
triptots2(pattern_counter); 

If  (HammOfst  >  CRITERIA) 
quadfets2(pattern_counter); 
if  (HammOist  >  CRITERIA) 
adapCcxitput_layer(patlefn_counter); 
if  (HamrnOM  >  CRITERIA) 

pattenT_teamed(patteoi_count8f)«=0; 


pattem_teamec(patt8fn_countef Js  1 ; 

. . . 

Apply  next  pattern 


) 


) 

r 

Store  weights  In  network  fBe  and  exit 

# 

save_weightJayersO ; 

) 

PROGRAM '.TEST3.C 
WRITTEN  BY:  W»am  Robinson 

DESCRIPTION:  Provides  user  with  a  test  menu  for  the  optical  neural  network. 


a^**«  #•*****••*• +«*•**•******#***•***#•*«••***# 


TEST  SET  function  looks  for  the  test  output  neuron. 

SSSS»«SMSSSSSSSSSSS>«S«»«SSStSSS»SSSStOSSSS»SHS>S<M>S»SS»< 

/ 

unsigned  int  test_set(kit  test_pattem_number) 

{ 

unsigned  int  actuaLvector; 
unsigned  Int  deferences; 
unsigned  Int  and_vecton 
actuaLvector  = 

(Output_Wnary_value{0]  •  1)  + 

(Output_Wnary_value{1)  *  2)  + 

(Output_blnary_value[2]  *  4); 

and_vector  *  actuaLvector  &  Letter  JestttesLpattem_number]; 

deferences  =  (and_yector  &  1)  +  ((and.vector  &  2)  »1)  +((and_vector  &  4) »  2); 

return  (differences); 

) 


TEST  NETWORK  function  test  network  for  given  Hamming  Distance  criteria. 
This  is  the  actual  test  function.  The  two  functions  following  this  one 
are  for  file  Vo. 

••***•»*•**•##*• *********** eft**********************************************  */ 

void  tesLnetwork(void) 

{int  tesLpattem; 

geLweighUayersO; 

ti(OPTICAL«1){ 

MOOE_00;  /*  Configure  interface  board  inside  386  computer  V 
ENABLE_WEIGHTS();  r  Allows  SLM  to  be  programmed  V 
WR1TEJLAYER1  _WEIGHTS0; 

WRITE^LAYER2_WEIGHTS0; 

) 


drscrO; 

prfotf("TESTING...Harnrning  Distance  Criteria  is:  %d\CRITERIA); 
gotoxyfl  J2);  9 

prirtrfTesting  Network  for  pattern:  "); 

/*****Begin 

for  (test_patlem»0;test_pattem<1  0;test_pattem++){ 

90kwy(3l^); 

dreolO: 

gotoxy(3l^): 

printff‘%(r,test_pattefn); 

/*******  Present  input  vector***********************  *•••***•****•  »**»•»•»•*•/ 

tf<(OP^CAL^l>{eS^,S^l8S^f>attem],T e^ms^tes*-Patteml): 

WRITE_INPUT_VECTOR(Test_isb{tesLpattem]. 

T est_msb{t8st_pattem]) ; 
reacLWddenl_binaiy(); 

WRITE_HIDDENt3INARY0: 

reacLriidden2_binary(); 

generatejoutpuLnewonsO: 

else  if  (SOFTWARE_VERSION=2){ 
slmulate_tayer1  .neuronsO; 

Simula  te_layer2_neurons0: 
generate_output_neuronsO; 

\else{ 

generate Jayer1_neurons0: 
generate Jayer2_neurons(); 
generate_outpuLneuronsO; 


} 


••******•• 


/*******Check  output  vector  *********** 
if(test_set(test_pattem)){ 
gotaxy(5,3+test_pattem); 
printffPattem  %d  recognizecT.tesLpattern); 


gotoxy(5,3+test_pattem); 

printf(*Pattem  %d  not  recognized*, tesLpattem); 

Apply  next  pattern 

} 

printf(*\n  Hit  any  key  to  continue"); 
getchO; 

) 

Hamming  Distance  criteria. 


void  one_test(voidK 

char  num[2); 
charcopy^]; 
chartemp{12]; 
drscrO; 

if  (system_seiectecl){ 
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KtoeUtM 

do{ 

prin  tf (*  Enter  Hamming  Distance  Training  Criteria  (between  0  and  3): "); 
scanfpW.&CRITERlA); 

}  while  ((CRITERIA  <  0)  I  (CRITERIA  >  3)); 
strcpy(temp,ln_name); 
ltoe(CRITERIA,num,10); 

8trcpy(copy.‘.00'); 
strcat(copy,num); 
strcatOn_name,copy); 
tesLnetworkO; 
strcpy(ln_name,  temp); 

) 

else{ 

printff Need  filename  to  get  weights^"); 

printffExit  to  MAIN  menu  and  select  FILE  menu  (hit  any  key)\n"); 

getchQ; 

) 

} 

eise{ 

printf(‘Need  to  select  systemNn"); 

printffExit  to  MAIN  menu  and  select  system  parameters  (hit  any  key)\n*); 

getchO; 

) 

} 

^****ee«***ft**«*** *********************************************************** 

MULTIPLE  TESTS  function  tests  network  for  a  user  prompted  starting  and 
ending  consecutive  Hamming  Distance  cdterias. 

****we***********e**#***e***********e**«******ee#*e*******#***«e**e*********^ 

void  multiple  Jests(void) 

{ 

char  num[2]; 
charcopy[5]; 
chartemp[12]; 
int  startfinish; 
drscrO; 

if  (system_selected){ 

WfeeUH 

do{ 

printffEnter  starting  Hamming  Distance  Training  Criteria  (between  0  and  2):  *); 
scanff%d",&start); 

}  while  ((start  <  0)  I  (start  >  2)); 
do{ 

printffEnter  ending  Hamming  Distance  Training  Criteria  (between  start  and  3):  "); 
scanff%d’,&finisb); 

}  while  ((finish  <=  start)  I  (finish  >  3)); 
for  (CRITERlAsStart;CRITERIA<Bfinlsh;CRITERIA-M-){ 
strcpy(temp,ln_name); 
ltoa(CRITERI  A.num,  1 0); 
strcpytcopy.VOO*): 
strcat(oopy,num); 
strcat(ln_name,copy); 
tesLnetworkO; 
strcpy(ln_name,temp); 
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) 

} 

etae{ 

pdntf(*Need  filename  to  get  weights'^*); 

printffExit  to  MAIN  menu  and  select  RLE  menu  (hit  any  key)\n"); 

getchO; 

} 

} 

eiso{ 

printf(*Need  to  select  systemNn"); 

printf(*Exit  to  MAIN  menu  and  select  system  parameters  (hit  any  key)\na); 
getchO; 

} 

geUt=0; 

J 

ya— tSSSte«»S»SSSSS»SiSSSttM>»SH»SStStMS*«HS**SMM*SSSS 

TEST  MENU  FUNCTION  prompts  user  for  training  options, 
••••a***********************************************************************/ 


void  test_menu(void) 
{ 

int  tesLchoice; 


clrscrO; 

printfOn"); 

printf  ("MNtOpticai  ANN  Test  Menu\n*); 
printf  (•  \na); 

printf  ("Vt  0.  Return  to  main  menuNn*); 
printf  ("U 1.  Single  testing; 
printf  ("Vt  2.  Multiple  testsNn"); 
printf  CVt->a); 

scant  (*%da  ,&test_cboice) ; 
switch  (tesLchoice)  { 
caseO:{ 

geLit=0; 

return; 

} 


case  1:  oneJestO; 
break; 

case  2;  multipte_tests(); 

break; 

defaultftreak; 

1 

)  while  (tesLchoice  1=  0); 

) 

^SMftMMSSMMMSSSSftSMMSSftMSMMSMSStSSSMftSStftMSHSMSftSSSMSSSMSMSMMSSSM 

PROGRAM:  DRIVER3.C 
WRITTEN  BY:  William  Robinson 

DESCRIPTION:  The  following  routines  provide  hardware  I/O  interface  to 
optical  ANN  and  host  computer.  I/O  is  performed  through  a  96  bit  DIGIO-96 
computer  interface  board  located  at  base  address  HEX  300  utilizing  an 
in-house  engineered  break-out  board,  and  by  a  48  channel  analog  to  digital 
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converted  board  at  base  address  HEX  320.  This  board  reads  32  optical  analog 
outputs  correspondtog  to  16  optical  outputs  for  layer  1  and  16  optical 
outputs  for  layer  2. 

/**•**“************•****“•****•••“•*****•“***•““**•*•*•******••••••*/ 

void  ADC JLAYER1  (void) 

{ 

unsigned  int  channel; 

Int  tow_byte,hlgh_byt8,adc_counts; 
unsigned  char  chedeEOC; 
double  analog; 

/••—Read  layer  1  channels****/ 
for  (channel  =  0;  channel  <  16;  channel-M-){ 
outportb(SET_CHANNEL, channel); 
oulportb(ADC_CONVERT,TRIGQER); 
do 
{ 

checK_EOC*lnportb(EOC); 
printf(*EOC  »  %u  \n*,checK_EOC); 

} 

while  (check_EOC  >  127); 

low_byte=fnportb(READ_LOW); 

hlgh_byte=lnportb(READ_HIGH); 

adc_oounts=((highJ>yte*16)  +  (tow_byta/16));  /TOTAL  ADC  COUNTS  7 
Hidden  1  _analog_value[cnannel]  =  adc_counts  *  .00122; 

} 

} 

void  ADC_LAYER2(void) 

{ 

unsigned  int  channel; 
int  towjbyte,hfoh J>yte£dc_counts; 
unsigned  char  checKJEOC; 
double  analog; 

/*****Read  layer  2  channels****/ 
for  (channel  a  16;  channel  <  32;  channel++){ 
outportb(SET_CHANNEL,  channel); 
outportb(ADC_CONVERT, TRIGGER); 
do 
{ 

checK_EOC=inportb(  EOC)  ; 
printffEOC  *  %u  \n\checK_EOC); 

} 

while  (checK_EOC  >  127); 
low_byte=lnportb{  R  EAD_LO W); 
high_byte=Jnportt)(READ_HIGH); 

adc_counts=((hlgh_byte*  1 6)  +  (low_byte/16));  /TOTAL  ADC  COUNTS  7 
Hidden2_analog_vaKje[channel-1 6]  =  adc.counts  *  .00122; 

} 


Setup  Digital  10  Interface  Board  in  computer,  board  base  address  is  Hex  300. 

***********  ft***************************************************************/ 


void  MODE_0(void) 

{ 

outportb(BASE+3, 1 28); 
outportb(BASE+7, 1 28); 
outportb(BASE+1 1,155); 
outportb(BASE+1 5. 1 37); 

} 

y***«t**«***SM***M*i«*S**i*tl 


I*  Mode  0  Output  7 
r  Mode  0  Output  V 
/*  Mode  0  Input  V 
/*  Mode  0  Input/Output  V 


Writes  to  sixteen  cascaded  HUGHES  4038  32-pixel  SLM  waveform  generator  chips. 
Actually  generates  pixel  "ON"  phases  which  get  either  amplified  or  attenuated 
by  programmable  gain  amplifiers. 

**•“***— *****************‘*‘*****‘***************************************/ 


void  ENABLE_WEIGHTS(void) 

{ 

inti; 
ch arj; 

outportb(BASE  +  5,0);  /*  Turn  on  SLM  pixels  V 
for  (fc=0;k=255;i++) 

{ 

outportb(BASE  +  4,LCD1_CLOCK  +  LCD2_CLOCK);  /*  Clock  it  in  V 
delay(l); 

outportb(BASE  +  4,0); 
delay(l); 

} 

/*  Load  data  into  chip  registers  7 
outportb(BASE  +  5,LCD1_LOAD  +  LCD2_LOAD); 

} 

/*••••*•*****••**•*•****••**•*•*****••**•*«*•**•***«**••***•*•**•••*•******* 


Write  the  input  vector  read  from  training  file  in  MR  2  routine. 


void  WRiTEJNPUT_VECTOR(unsigned  char  Isb, unsigned  char  msb) 
{ 

unsigned  char  tmp[8]; 

r  Input  LED’s  are  reversed  In  order,  swap  order  *•*»*»**••»•*»**••*••*** 
bito  — bit7 


bitl  —  bite 

blt2  —  bits 

bit3  —  btt4 
bft4  —  bit3 

bits  —  bit2 
bite  —  bitl 

bit7  —  bitO 


tmp[0]  =  (msb&64)»6; 
tmp[1]  s  (msb&32)»4; 
tmp[2]  s  (msb&16)»2; 
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tmp{3] » (msb&8); 

tmpl4]  *  (msb&4)«2; 

tmptsj » (msb&2)«4; 

tmp{6]  =  (msb&1}«6; 

tmp{7]  =  1;  /*  Bias  neuron  V 

msb  =  tmp{0]ltmp(1]ltmp[2]ltn^3]ltmp[4]ltmp[5]ltmp[6]ltmp(7]; 

tmp{0] » (lsb&128)»7; 
tmp(1]  a  (lsb&64)»5; 
tmp{2]  =  (lsb&32)»3; 
tmp{3]  a  (lsb&16)»1; 
tmp{4j  =  (lsb&8)«1; 
tmp{5]  a  (lsb&4)«3; 
tmpt6)  a  (lsb&2)«5; 
tmp{7]  a  (lsb&1)«7; 

Isb  a  tmp[0]ltmp{1]ltmpt2]ltrnpl3]ltmp[4]ltmp{5]ltmp(6]ltmp[7]; 

outportb(ILED_LO,lsb); 

outjx>rtb(ILED_Hltmsb); 


Write  the  output  from  hidden  layer  as  Input  to  output  layer  only  when  using 
all  hardware  thresholding. 

**••*****•********•••••*••*•••••**•*•**•••••**••*•******##**••*1 *******•*•#*•/ 
void  WRITE_HI  DDEN 1  _BIN  ARY(vold) 

{ 

unsigned  char  tmpt81.msb.lsb; 

^**ft*MtMMM*MMS***ii««**«***MHIiM«****S*tM*«***«iiftMfttt*fl*tft**SS*4 

Read  the  hidden  layer  outputs 

msb  a  lnportb(HOUT_HI); 

Isb  =  inportb{HOUT_LO); 

r  Input  LED’s  are  reversed  In  order,  swap  order  •••••••••••••••••••••••• 

bito  —  Wt7 
bitl  —  bit6 
bft2  —  Wt5 
bit3  —  blt4 
Wt4  — bit3 
bit5  —  btt2 
blt6  —  bill 
blt7  —  bttO 


tmp{0]  a  (msb&64)»6; 
tmp{1]  a  (msb&32)»4; 
tmp{2]  a  (msb&16)»2; 
tmp{3j  a  (msb&8); 
tmp{4]  a  (msb&4)«2; 
tmptsj  a  (msb&2)«4; 
tmp{6]  a  (msb&1)«6; 
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tmp(7]  =  1;  r  Bias  neuron  V 


msb  a  tmp[0]ltmp(1]ltmp{2]ltmp[3]ltmp{4]ltmp{5]ltmp{6]llmp)C7]; 

tmp{0]  =  (lsb&128)»7; 

tmpjl]  =  (lsb&64)»5; 

tmp{2]  =  (lsb&32)»3; 

tmp{3j  a  (lsb&16)»1; 

tmp{4]  =  (lsb&8)«1 ; 

tmp{5]  a  (lsb&4)«3; 

tmp{6]  =  (lsb&2)«5; 

tmp[7]  =  (lsb&1)«7; 

tsb  a  tmp{0]ltrnp[1]ltrnp{2]ltmp[3]ltmp{4]ltmp[5]ltrnp[6]ltmp[7]; 


outportb(HLED_LO,msb); 

outportb(HLED_Hl.lsb); 

} 

/****•***************“********* 


Write  the  output  from  hidden  layer  as  input  to  output  layer  when  using 
ADC  board. 


void  PRESENT_HIDOEN1_VECTOR(vold) 

{ 

Int  I; 

unsigned  char  tmp[8],msb,lsb; 
for  (l=0;l<8;l++){ 

H  (Hidden  1  _binary_value[i]) 

tmpp]ai; 

else 


tmp(i]=0; 

tmptn=(tmp(i] « I); 

} 

tmp{7]  =  1;  /*  Bias  neuron  V 

Isb  a  tmp[0]ltmp{1]ltmp{2]ltmp[3]ltmp[4]ltmp(5]ltmp[6]ltmpf7]; 
for  (fc=0;l<8;l++){ 
if  (Hidden  1_Wnary_value[l+8]) 
tmpnjal; 
else 

tmpp]=0; 

tmp[i]=(tmp(i]«l); 

} 

tmp{7]  =  1;  r  Bias  neuron  7 

msb  a  tmp(0]ltmp(1]ltmp(2]ltmp[3]ltmp(4]ltnip{5]ltrnp{6]ltmp[7]; 

outportb(HLED_LO,lsb); 

outportb(HLED_HI,msb); 


This  routine  writes  the  values  of  weights  found  in  WEIGHT_LAYER1[16][16] 
to  the  corresponding  hardware  programmable  gain  amplifier.  The  programmable 
gain  amplifiers  provides  a  gray  level  driving  signal  to  the  weight  mask. 


void  WRITE_LAYER1_WEIQHTS(voW) 


Int  IJ,k,n; 

unsigned  char  temp,a,b. 


rK)ARD1[32],  r  Board  1  registers  V 

BOARD2(32],  r  Board  2  registers  7 

BOARD3(32J,  /*  Board  3  registers  V 

BOARD4[32],  /•  Board  4  registers  V 

BOARD5{32],  /*  Board  5  registers  V 

BOARD6{32],  /*  Board  6  registers  V 

BOARD7132],  /*  Board  7  registers  V 

BOAR 08(32],  r  Board  8  registers  V 

OUTWEIQHTS 1(256],  /*  Output  array  for  writing  to  SLM1  V 

DO  =  1,  /*  Typecasting  numbers  for  bft  shifting  V 

01  >2. 

02  =  4, 

03-8, 

04-16, 

05  =  32, 

D6-64, 

D7-128; 

^SSSSMMSSSMSMMeMSSeMSHMSSeSSSMeMMSHeeSHSSSMSMHSMMeSMSiM 

Convert  the  weights  and  biases  into  usable  forms  for  the  optical  weights 

for  0  »  0;  i  <  15;  i++) 
for  (j  =  0;  |  <  15;  ]++)  { 
if  (WeighUayerl  (IJfl  <  -60) 

OptlcaLlayerlliJj]  - 150; 
else  lf(Weight_layer1  [i]ffl  >  60) 

OpticaLlayeii[i]Q]  -  30; 

q|80 

OpticaLlayeripjO]  -  (int)(-WelghtJayert  (!}Q])+90; 

} 

Mkf  transmission  for  column  16,  will  be  used  for  active  thresholding. 

for  fl  =  0;  j  <  15;  J++) 

OptlcaUayer1[15]0] »  90; 


. ^  conespodinfl  weight  to  sim  driver  board 

for  (i=0,n=0;k=3;l++) 
for  O=0^c=7^-f,r»++) 

BOARD1[n]=  (unsigned  char)0pticaUayer1  [i]Q]; 
for  (b4,n=0;l<=7;k+) 
for  (j=0;j<=7;J++,n++) 

B0ARD2(n]=  (unsigned  charJOpticalJayerl  (!](]]; 
for  (U8,n-0;k=1 1  ;l++) 
for  (J=0J<=7;}++,n++) 

BOARD3{n)=  (unsigned  char)OptJcal_iayer1  [i]0]; 
for  (M2,n-0^c-15;i++) 
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for  O»0'j<»7'1}++  ,!>++) 

B0ARD4{n]®  (unsigned  char)0pticaUayer1[i]Q]; 
for  (b*0jVB0;io«3;i++) 
for  n=0-Us.7-U+  nui 

B0ARD5{n>>  (unsigned  char)OpticaLlayer1  [H[1 5-fr 
for  (fc«4,n®0;k=7;i+-f) 
for  (J=0;J<*7;fH-,n++) 

BOARD6(nJ=  (unsigned  char)OptJcalJayer  1  PHI  5-fl; 
for  (i=8,n=0;k*1 1  ;U-+) 
for  0=K)J<a7^++,n++) 

BOARD7[n]*  (unsigned  char)Optical_layer1  [ifll  5-fl; 
for  0=12, n=0;k=15;M 
for  (J=0J<=7J++.n++) 

BOARD8(nJ=  (unsigned  char)Optical Jayerl  [i](1 5-fl; 

/**Fix  order  error  for  SLM1  *7 
for  (n=8;n<=15;n++) 

{ 

temp=BOARD1  [n];  BOARD1  [n]=BOARD1  [n+1  ];  BOARD  1[n+1)=temp; 
temp=BOARD2Jn];  BOARD2[n)=BOARD2[n+1];  BO  ARD2[n+1  ]=temp; 
temp=BOARD3(n];  BOARD3(n]=BOARD3{n+1  j;  BOARD3(n+11=temp; 
tern  p=  BOAR  D4{n];  BOARD4[n)=BOARD4[n+1  ];  BOARD4(n+1  J=temp; 
temp=BOARD5[n];  BOARD5[n)=BOARD5[n+1];  BOARD5[n+1]=temp; 
temp=BOARD6[n];  BOARD6[n]=BOARD6{n+1  ];  BOARD6[n+1]=temp; 
temp=BOARD7[n];  BOARD7[n]=BOARD7[n+1  ];  BOARD7[n+1}=temp; 
temp=BOARD0(n];  BOARD8[n]=BOARD8(n+1];  BOARD8[n+1]=temp; 
n++; 

f 

for  (n=24;n<=31  ;n++) 

{ 

temp=BOARD1[n];  BOARD1  [n]=BOARD1  [n+1  ];  BOARD1  [n+1  ]=temp; 
temp=BOARD2(n];  BOARD2[n]=BOARD2(n+1  ];  BO ARD2[n+ 1  ]=temp; 
temp=BOARD3(n];  BOAR  D3[n]=BOAR  D3{n+ 1  ];  BOARD3(n+1]=temp; 
temp=BOARD4[n];  BOARD4[n}=BOARD4[n+1];  BOARD4ln+1]=temp; 
temp=BOARD5{n];  BOARD5[n]=BOARD5[n+1  ];  BOARD5[n+1]=temp; 
temp=BOARD6(n];  BOARD6{n}=BOARD6{n+1  ];  BOARD6{n+1]=temp; 
temp=BOARD7[nj;  BOARD7(n)=BOARD7(n+1];  BOARD7[n+1  }=temp; 
temp=BOARD8[n];  BOARD8[n)=BOARD8(n+1  ];  BOARD8[n+1]=temp; 


St-*'  ’1 

|  (7, 

Rv.iL 


/**  Now  set  up  ordinal  serial  sequence  and  concatenate  ail  8  *7 
/**  board  data  Ones  to  be  docked  out  *7 

. . . . . 

The  following  defines  driver  board  data  fine  to  output  port  of  interface 

BOARD  # _ BIT  POSITION _ BIT 

BOARD  1  0  1 

BOARD2  1  2 

BOARD3  2  4 

BOARD4  3  8 

BOARDS  4  16 


168 


BOARDS  5  32 

BOARD7  6  64 

BOARD8  7  128 

for  (n*0,fc=31  ;n<=255;n++,l-) 

{ 

OUTWEIGHTSl  [n]  =  ((B0ARD1P1&D7)»7)  I  ((BOARD2fl]&D7)»6) 
I  ((BOARD3[J]AD7)»5)  I  ((BOAJ=tD4[l]&D7)»4) 

I  ((B0ARD5P1&D7)»3)  I  ((BOARD6[i]&D7)»2) 

I  ((BOARD7(I]&D7)»1)  I  (BOARD8{i]&D7); 

n++; 

OUTWEIGHTSl  (nj «  «BOARD1[l)&D6)»6)  I  ((BOARD2[i]&D6)»5) 
I  ((BOARD3(i]&D6)»4)  I  ((BOARD4[i]&D6)»3) 

I  ((BOARD5[i]&D6)»2)  I  ((BOARD6[i]&D6)»1) 

I  (BOARD7{l]AD6)  I  ((BOARD6{i]&D6)«1); 

n++; 

OUTWEIGHTSl  [n] «  ((BOARD1[i]&D5)»5)  I  ((BOARD2(i]&D5)»4) 
I  ((BOARD3fl]&D5)»3)  I  ((BOARD4[i]&D5)»2) 

I  ((B0ARD5P1&D5)»1 )  I  (BOARD6[i]&D5) 

I  ((BOARD7in&D5)«1)  I  ((BOARD8[i]&D5)«2); 

n++; 

OUTWEIGHTSl  [n]  =  ((B0ARD1(i]&04)»4)  I  ((BOARD2pl&D4)»3) 
I  ((B0ARD3P1&D4)»2)  I  ((BOAR  D4  [!]&[>)»  1 
I  (BOARD5{l]&D4)  I  ((BOARD6[i]&D4)«1) 

I  (BOARD7[i]&D4)«2)  I  ((BOARD8[I]&D4)«3); 

n++; 

OUTWEIGHTSl  [n]  =  ((B0ARD1(i]&D3)»3)  I  ((BOARD2[i]&D3)»2) 
I  ((BOARD3[i]&D3)»1)  I  (BOARD4[i]&D3) 

I  ((BOARD5[i]&D3)«1 )  I  ((BOARD6(il&D3)«2) 

I  ((BOARD7p]&D3)«3)  I  ((BOARD8{i]&D3)«4); 

n++; 

OUTWEIGHTSl  [nj  =  U60ARD1[I]&D2)»2)  I  ((BOARD2[IJ&D2)»1) 
I  (BOARD3(i]&D2)  I  ((BOARD4[l}&D2)«1) 

I  ((BOARD5[i]&D2)«2)  I  ((BOARD6[i]&D2)«3) 

I  ((BOARD7P]&D2)«4)  I  ((BOARD8[i]&D2)«5); 

n++; 

OUTWEIGHTSl  [n]  =  ((BOARDip]&D1)»1)  I  (BOARD2W&D1) 

I  ((BOARD3{i]&D1)«1)  I  ((BOARD4[i]&Dl  )«2) 

I  ((BOARD5[i]&D1  )«3)  I  ((BOARD6{i]&Dl)«4) 

I  ((BOARD7(l]&D1)«5)  I  ((BOARD8{i]&Dl)«6); 

r>++; 

OUTWEIGHTSl  [n]  =  (BOARD1[i]&DO)  I  ((BOARD2[i]&DO)«1) 

I  ((BOARD3{i]&DO)«2)  I  ((BOARD*  «3) 


I  {(BOAR05{i]&D0)«4)  t  ((BOARD6(I}&DO)«5) 

I  ((BOABO7II]&D0)«6)  I  ((BOARD6(I)&DO)«7); 

} 

/**  Now  dock  out  the  weights  serially  *7 
/**  First  set  up  load  Rnes  high  so  data  can  be  docked  in  **/ 
outporib(BASE  +  5.LAYER1  JLOAD); 
for  (W)J=0'J<=15J++) 

{ 

r  First  dock  out  SSBIT*/ 

outportb(BASE  +  4,0); 

outportb(BASE  +  12.SSBIT); 

outportb(E  \SE  +  4.LAYER1  .CLOCK); 

r  Now  dock  out  next  16  bits  in  the  folowing  order  V 

r  MSB1 _ LSB1  MSBO . LSBO  V 

for  (k=0;k<=15;k++) 

{ 

outportb(BASE  +  4,0); 
outportb(BASE  +12,OUTWEK3HTS1[i]); 
outportbfBASE  +  4.LAYER1  .CLOCK); 
i++; 

) 

) 

r  Now  load  data  in  programmable  potentiometer  registers  *7 
outportb(BA3E  +  5, 0); 

} 

/******«**•*******•***•*•****•***•«**••****•*****«****•****«*•****••*•••*** 


This  routine  writes  the  values  of  weights  found  in  WEIGHT_LAYER1[16|16] 
to  the  corresponding  na/oware  programmable  gain  amplifier.  The  programmable 
gain  amplifiers  proves  a  gray  level  driving  signal  to  the  weight  mask. 

void  WRITE_LAYER2_WEIGHTS(void) 

{ 

int  IJ.k,n; 
unsigned  char 
temp.a.b, 

BOARD9(32],  /*  Board  9  registers  7 

BOARD  10(32],  .**  Board  10  registers  7 

BOARD1 1(32],  r  Board  1 1  registers  7 
BOARD  12(32],  /*  Board  12  registers  7 

BOARD  13(32],  /*  Board  13  registers  7 

BOARD14[32],  /*  Board  14  registers  7 

BOARD15{32],  /*  Board  15  registers  7 


BOARD  16(32],  /*  Board  16  registers  7 

OUTWEK3HTS2(256],  r  Output  array  for  writing  to  SLM2  V 
DO- 1,  r  Typecasting  numbers  for  bit  shifting  7 

D1-2. 

D2-4. 

D3-8. 

D4-16, 

DS  -  32, 

D6-64, 

D7*128; 

Convert  the  weights  and  biases  into  usable  forms  for  the  optical  weights 

*«*M*MtMM*MM*MM*M«MMft*«****t***«**HMftMM*M**«M«MM«MMtM*^ 

for  (i  *  0;  i  <  15;  I++) 

for(l  =  0;l<15;fr+){ 
if  (WeighLiayei2(iKD  <  -60) 

Optfcal_layef2(i]Q]  *  150; 

else  K(WeighUaye*2(i]fll  >  60) 

OpticaUayer2(nQ]  =  30; 
else 

Optical Jayer2[i]01  =  (lnt)(-WeighLlayer2p]0])+9O; 

) 


Mid  transmission  for  column  16,  will  be  used  for  active  thresholding. 
***•*•*•*****••*•***•********•***•***•****•*******••**•##******•**♦* ****••*/ 
for  0  =  0;  J  <  15;  J++) 

Optteal_layer2[1 5][]]  a  90; 

/********************•******•****•*•••*•*•**••••*•••*•••**•*»—**»**»*»•»*** 
Assign  correspoding  weight  to  sJm  driver  board 

ft******************** ft*****************************************************^ 

for  (fc=0,n=0;k=3;l++) 
for  (J=0;]<=7;fM-,n++) 

BOARD9(n)a  (unsigned  char)Optical_layer2{i][j] ; 
for  (i=4,n=0;l<=7;l++) 
for  Q=0;J<=7;j++,n-M-) 

BOARD10(n]=  (unsigned  char)OpticalJayer2[f][fl; 

for  (Is8,na0;kal  1  ;l++) 
for  ()=0J<=7;j++,n-H-) 

BOARD11[n)=  (unsigned  char)OpticaLlayer2(l][n; 
forO=12,n=0;k=15;l++) 
for  0=OJ<a7;j++,n++) 

BOARD12(n}=  (unsigned  char)Optical_layer2[i][]]; 
for  (U0,n=0;k=3;k-t-) 


for  0-O*J<-7J^,iv++) 

BOARD13(n]B  (unsigned  char)0pticaljayef2(1|1 5-fl; 
for  (W,nxO*c7;M 
for  O*0^c*7^f+4VH-) 

BOARD  14{n}=*  (unsigned  char)OpticaLlayef2p](15-fl: 
for  (WLn»0;k*1 1  ^++) 
for  0»Ojo»7J++^H-f) 

BOARD15[n}>  (unsigned  char)Optical_layer2(iX1 5-fl; 
for  0=a12,n=O;k»15;i++) 
for  Q=0J<=7^M-.rvM-) 

BOARD  16(n)=  (unsigned  char)Optical_layer2(I](1 5-fl; 

/**Fbc  order  error  for  SLM2  **/ 
for  (n*8;n<*15;n++) 

{ 

temp=BOARD9(nJ;  BOAR  D9(n]=  BOAR  D9(n+1  J;  BOARD9[n+1  Hemp; 
temp=BO ARD  1 0[n];  BOARD10(n>=BOARD10(n+1];  BOARD1 0(n+1  }=temp; 
temp=BOARD1 1  [n];  BOARD  1 1  [n>=BOARD1 1  [n+IJ;  BOARD11[n+1}=temp; 
temp-BOARD12(nJ;  BOARD12(n]=BOARD12Ir>+tJ;  BOARD1 2(n+1  }=temp; 
temp=BOARD13(n];  BOARD13{n)=BOARD13(r>+1J;  BOARD13[n+1J=temp; 
temp=BOARD  1 4[n];  BOARD14[n}=BOARD14[n+1];  BOARD1 4[n+1  }=temp; 
temp=BOARD1 5{n];  BOARD15[n]=BOARD15{n+1];  BOARD  15{n+1}=temp; 
tempsBOARD16[n];  BOARD16[n]=BOARDiein+1];  BOARD1 6(n+1  }=temp; 
n++; 

} 

for  (n=24;n<=31  ;n++) 

{ 

temp=BOARD9(n];  BOAR  D9(n}=  BOAR  D9(n+1  J  BOARD9[n+1  Hemp; 
temp=BOARD10(n];  BOARD10(n}=BOARD10(r>+1];  BOARD10(n+1}=temp; 
temp=BO  ARD 1 1  [n];  BOARD1 1  [n)=BOARD1 1(0+1};  BOARD1 1  [n+1  }=temp; 
temp=BOARD12(nl;  BOARD12[n]=BOARD12[n+1];  BOARD1 2[n+1  J=temp; 
temp=BOARD13{n];  BOARD  1 3{n}=BOARD  1 3{n+ 1  ];  BOARD1 3(n+1  }=temp; 
temp=BOARD14(n];  BOARD14[n]=BOARD14{n+1];  BOARD14[n+1J=temp; 
temp*BOARD1 5(n];  BOAR  D 1 5{n  J=BOARD  1 5[r>+ 1  ];  BOARD15(n+1]=temp; 
tempxBOARDI  6{n];  BOARD16{n}=BOARD16[n+1];  BOARD16(n+1J=temp; 
n++; 

} 

r  Now  set  up  ordhal  serial  sequence  and  concatenate  all  8  *7 
r  board  data  lines  to  be  docked  out  **/ 

The  folowing  defines  driver  board  data  Bne  in  output  port  of  interface 

_ BOARD  # _ BIT  POSITION _ BIT  VALUE _ 

BOARD8  7  128 

BOARD9  0  1 


BOARDIO  1  2 

B0ARD11  2  4 

BOARD  12  3  8 

BOARD  13  4  16 

BOARD14  S  32 

BOARD 15  6  64 

BOARD  16  7  128 

for  (n=0,fc=31  ;n<=255;n++,l~) 

{ 

OUTWEIGHTS2[n]  a  ((BOARD9{l]iD7)»7)  I  ((BOARD10[i]&D7)»6) 
I  ((BOARD1 1  PJ4D7)»5)  I  ({BOARD  1 2(04  D7)»4) 

I  ((BOARD1 3[Q&D7)»3)  I  ((BOARD14{i]&D7)»2) 

I  ((BOARD1 5[i]&D7)»1 )  I  (BOARD  16(1]&D7); 

n++; 

OUTWEIGHTS2[n]  a  ((BOARD9{i]&D6)»6)  I  ((BOARD10P]4D6)»5) 
I  ((BOARD  1 1  (i]4D6)»4)  I  ((BOARD  12(04D6)»3) 

I  ((BOARD1 3(i]&D6)»2)  I  ((BOARD14[i]4D6)»1) 

I  (BOARD1 5{i]&D6)  I  ((BOARD16fl)4D6)«1); 

n++; 

OUTWEIGHTS2[n]  a  ((BOARD9(i]4D5)»5)  I  ( (BOARD  10(i}&D5)»4) 

I  ((BOARD1 1  [i]&D5)»3)  I  ((BOARD  12[i]&D5)»2) 

I  ((BOARD1 3[Q&D5)»  1 )  I  (BOARD14IQ&D5) 

I  ((BOARD1 5{I]&D5)«1 )  I  ((BOARD16(I]&D5}«2); 

n++; 

OUTWEIGHTS2(n]  a  ((BOARD9(1]&D4)»4)  I  ((BOARD10PJ&D4)»3) 
I  ((BOARD  1 1[?]&D4)»2)  i  ((BOARD12[i]&D4)»1) 

I  (BOARD13[i]&D4)  I  ((BOARD  14(i]&D4)«1) 

I  ((BOARD15[i]&D4)«2)  I  ((BOARD16[i]&D4)«3); 

n++; 

OUTWEIGHTS2[n]  =  ((BOARD9(i]&D3)»3)  I  {(BOARD10(1]&D3)»2) 
i  ((BOARD1 1  P1&D3)»  1 )  I  (BOARD1 2[i]&D3) 

I  ((BOARD1 3{Q&D3)«1 )  I  ((BOARD14[i]&D3)«2) 

I  ((BOARD1 5[i]&D3)«3)  I  ((BOARD16{i]&D3)«4); 

n++; 

OUTWEIGHTS2(n]  a  ((BOARD9[f]&D2)»2)  I  ((BOARD10P]&D2)»1) 
I  (BOARD  11[i]&D2)  I  ((BOARD12(]]&D2)«1) 

I  ((BOARD1 3{i]&D2)«2)  I  ((BOARD14[i]8D2)«3) 

I  ((BOARD1 5[i]&D2)«4)  I  ((BOARD16(i]&D2)«5); 

n++; 

OUTWEIGKTS2[n]  a  ((BOARD9(i]&D1)»1)  I  (BOARD10[Q&D1) 

I  ((BOARD1 1  [Q&D1  )«1 )  I  ((BOARD1 2(i]&D1  )«2) 

I  ((BOARD1 3{i]&D1  )«3)  I  ((BOARD14p)&D1)«4) 

I  ((BOARD1 5[i]&D1  )«5)  I  ( (BO ARD 1 6(0401  )«6); 


0UTWEJGHTS2fn]  >  (BOARD9(i]4DO)  I  ((BOARD  10(f|4D0)«1) 
I  ((BOARD1 1  [i]&D0)«2)  I  ((BOARD  12(I]&D0)«3) 

I  ((BOARD  1 3(n&D0)«4)  I  ((BOARD  14[i]&D0)«5) 

I  ((BOARD1 5{i]&D0)«6)  I  ((BOARD1 6(I]&D0)«7); 

) 

/**  Now  dock  out  the  weights  serially  **/ 

/•*  First  set  up  load  Hnes  high  so  data  can  be  clocked  in  *7 
outportb(BASE  +  5,LAYER2_LOAD); 
for  (l=0J=0i<=15;H+) 

{ 

/*  First  dock  out  SSBIT  7 
outportb(BASE  +  4,0); 
outportb(BASE  +  13.SSBIT); 
outportb(BASE  +  4,LAYER2_CLOCK); 

/*  Now  dock  out  next  16  bits  in  the  following  order  7 

r  MSB1 _ LSB1  MSBO _ LSBO  7 

for  (k=0;k<=15;k++) 

{ 

outportb(BASE  +  4,0); 
outportb(BASE  +1 3,OUTWEIGHTS2[l]); 
outportb(BASE  +  4,LAYER2_CLOCK); 
i++; 

1 

) 

/**  Now  load  data  In  programmable  potentiometer  registers  *7 
outportb(BASE  +  5, 0); 

) 

^•***HMi***MSMSS*M*MSMS*Mftft**S*M********4**tftMH***«*t**t*M**MS^ 

void  LEDS_OFF  (void) 

{ 

outportb  (ILED_LO,OxOO); 
outportb  (ILED_HI,0x00); 
outportb  (HLED_LO,OxOO); 
outportb  (HLED_HI,0x00); 


PROGRAM :  FILE.C 

WRITTEN  BY:  William  Robinson 

DESCRIPTION:  Provides  simple  file  I/O  menu. 

**•****•*«*****•***•******«•*****••***••••••*••*•*•***••**********«***•*•*•*/ 


**«**t**t*t*t 


SAVE  FILE  FUNCTION  prompts  user  for  network  output  file  base  name. 
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void  save  Jle(void) 

{ 

printffNn  Enter  name  of  output 
scanff%12s\  out_name); 
save_ifc=1; 

} 


GET  FILE  FUNCTION  prompts  user  for  network  output  fRe  base  name. 


******************* **************/ 


voWget_«e(voW) 

{ 

prlntffln  Enter  name  of  input  file:  *): 
scanff%12s\  ln_name); 

geUt-i; 

} 


FILE  MENU  FUNCTION  prompts  user  for  network  files  used  for  input  and/or 
output 

•******•••#**«*«*•••*#•#*•#•*•*#•**•*«•••******««*#***•*#«*•«••***• *#*##««*«y 

void  ffle_menu(void) 

{ 

int  file_choice; 

y****************************************************************** *********** 

Display  the  ANN  Menu 

******M*********************************************ft********«***************^ 

do  { 


drscrQ; 

printf("\n‘); 

print!  ("VfitOptical  ANN  File  MenuVn'); 
print!  ("  \n“); 

print!  f\t  0.  Return  to  main  menu\n*); 
printf  (*\t  1 .  Save  weights  filename^'); 
printf  CM  2.  Get  weights  file  for  testing^"); 
printf  ftt  3.  Get  pattern  file<s)  for  trainingtH"); 
printf  ("Vt->«); 
scanfC%(f,&file_choice); 
switch  (file_choice)  { 
case  0:  return; 
case  1:  saveJileO; 
break; 

case  2:  gptJileO: 


break; 


3:  printf(*Optk>n  is  not  avaBable  at  this  time\n"); 


break; 


detaufcbreak; 


PROGRAM  :TRA1N.C 
WRITTEN  BY:  Wlffiam  Robinson 
DESCRIPTION:  Provides  user  with  a  training  menu. 


* . ********7 

******ee**a*eeeeaeeeeeeeaea**#*ee*eeeea*eee*ewaee 


ONE  RUN  function  trains  network  for  a  single  Hamming  Distance  criteria. 

•***a*e***«***+*#**««*«*ft«a««a*a*eaaft***eaeaeeaeeee**e*ee#*aeeee*e****eeeaee^ 


void  one_run(void) 

{ 

char  num(2]; 
charcopy[5]; 
char  temp(12]; 
clrscrO; 

If  (system_selected){ 
if  (savejt){ 
do{ 

printf(*Enter  Hamming  Distance  Training  Criteria  (between  0  and  4  inclusive):  *); 
scanf(*%d\&CRiTERIA); 

}  while  ((CRITERIA  <  0)  I  (CRITERIA  >  4)); 

strepy  (temp.out.name) ; 

ftoa(CRITERIA,num,10); 

strcpy(copy.\00*); 

strcat(copy,num); 

strcat(out_name,copy); 

if  (OPTICAL)  op«cal_ann(); 

else  if(SOFTWARE_VERSION=2)  simulateO; 

else  normalJrainingO; 

strcpy(out_name,temp); 

) 

else{ 

printf(’Need  filename  to  save  weigbtsVT); 

printf(’Exit  to  MAIN  menu  and  select  FILE  menu  (hit  any  key)\n*); 

getchO; 

} 

} 

else{ 

printf(*Need  to  select  systemNn"); 

printf('Exit  to  M/UN  menu  and  select  system  parameters  (hit  any  key)\n'); 
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getehO; 

) 

} 

MULTIPLE  RUN  function  trains  network  for  a  user  prompted  starting  and 

ending  consecutive  Hamming  Distance  criterias. 

void  muitipte_runs{void) 

{ 

char  num(2]; 
char  copy{5]; 
chartemp(12]; 
int  starLfinish; 
clrscrO; 

if  (system_setected){ 

If  (savejt}{ 
do{ 

printfCEnter  starting  Hamming  Distance  Training  Criteria  (between  0  and  3  inclusive):  '); 
scanf(*%d‘,&start); 

}  while  ((start  <  0)  I  (start  >  3)); 
do{ 

printfCEnter  ending  Hamming  Distance  Training  Criteria  (between  start  and  4  inclusive):  "); 
scanf(*%d‘,&finish); 

)  while  ((finish  <=  start)  I  (finish  >  4)); 
for  (CRITERIA=start;CRnrERIA«=finlsh;CRITERIA++){ 
strcpy(temp,out_name); 
itoa(CRITERI  A,num,1 0); 
strcpy(copy,\00‘); 
strcat(copy,num); 
strcat(out_name,copy); 
if  (OPTICAL)  optical_ann(); 
else  if(SOFTWARE_VERSION=2)  simulate^ 
else  normaLtrainingO; 
strcpy(out_name,temp); 

) 


printffNeed  filename  to  save  weights^*); 

printffExit  to  MAIN  menu  and  select  FILE  menu  (hit  any  key)\n*); 

getehO: 

) 

} 

else{ 


printf(aNeed  to  select  system\n"); 

printf  (‘Exit  to  MAIN  menu  and  select  system  parameters  (hit  any  key)\n'); 
getch(); 

} 

savejt=0; 

} 

/***“***************“"*********“**********************"****************** 

TRAIN  MENU  FUNCTION  prompts  user  for  training  options. 

void  train_menu(void) 

( 

int  tratn_choice; 

Display  the  ANN  Menu 
do{ 

clrscrO; 

printfC^n"); 

printf  ("MMOptlcal  ANN  Train  Menu\na); 
print/  ('  \na); 

printf  ("\t  0.  Return  to  main  menu\n*); 
printf  ("U 1.  Single  run\na); 
printf  (*\t  2.  Multiple  runs\na); 
printf  (a\t->a); 

scanf(*%d\&traln_choice); 
switch  (train_choice)  { 
case  0:  { 
saveJfcsO; 
return; 

} 

case  1:  one_run(); 
break; 

case  2:  multiple_runs(); 
break; 

default:break; 

} 

}  while  (train_choice  1=  0); 

} 

/“********"**************“*'**************“********“ . . 

PROGRAM:  uniform.c 
WRITTEN  BY:  Danny  Shelton 

DESCRIPTION:  u32  generates  a  32  bit  random  number  based  Seedl  and  Seed2 
parameters  located  In  ainclude3.ca.  This  function  is  called  by  uniform_dlst() 
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which  takes  two  numbers  and  generates  a  random  number  between  these  two 
numbers.  The  resulting  numbers  are  used  to  initialize  the  weight  arrays 
In  the  MR2  training  algorithm. 

double  u32(void)  { 

I*  32  bit  */ 
long  Int  z.k; 
k  =  Seedi/53668; 

Seedl  =  4001 4*(Seed1-k*53668)-k*1 2211; 
if  (Seedl  <0)  Seedl  +=  2147483563; 
k  =  Seed2/52774; 

Seed2  =  40692*(Seed2-k*52774)-k*3791 ; 
if  (Seed2<0)  Seed2  +=  2147483399; 
z  »  Seedl -Seed2; 
if  (z<1)Z+=  2147483562; 
return  (z*4.665613e-10); 

} 

double  uniform_dist(double  a.  double  b) 

{ 

Seedl  =  rand(); 

Seed2  =  rand(); 
retum(a+(b-a)*u32()); 

) 

^*•****4 ****•***«****••*•**•*•*••*•**•*••••***••••*«*•**••*•*•*•*•••*•*••*••* 

FILE:  RANK.C 

Borrowed  from  Numerical  Recipes  in  C  (see  References) 

DESCRIPTION:  Ranks  an  array  of  N  values  in  ascending  order.  Used  in 
Madaline  Rule  2  Training. 

A*****#*********************************************************************/ 


INDEXX  function  produces  an  array  pointers  which  indexes  the  values  passed 
in  ascending  order. 

void  lndexx(int  n,  double  arrinQ.int  indxQ) 

{ 

Int  I.J.ir.Indxtl; 
float  q; 

for  (j=l;  J  <=  n;  j++) 

IndxQH; 

l=(n  »  '  +■  1; 
lr=n; 

for  ( ; ; )  { 
if(l>1) 


q=arrin((lndxt=incb<H])]; 

eise{ 

q=anin((indxt=indx(lr])]; 
indxpr)sindx[1]; 
if  (~|r  =  1){ 
lndx[1]=indxt; 
return; 

1 

} 

M; 

H«i; 

while  (j  <=ir)  { 

if  (J  <  Ir  &&  arrfn[indx[fl]  <  arrinpndxQ+1  ]])  j++; 
if  (q  <  arrin[indxQ]])  { 

IndxIQslndxQ]; 

J  +=  (H): 

} 

elsej=ir  + 1; 

} 

fndx[i]=lndxt; 

} 

} 

^••** •**•***•**•*« *****••*«***«***••*•*****«**•••*••*•**••***••**•**********« 

RANK  function  takas  the  index  array  and  converts  it  to  a  ordered  rank  array. 

**t***i**HM*MS**M*iSlM*MS*******i****M***M*M*«t*ftMft***MH**t*****Y 

void  RankQnt  n.lnt  indxQ.int  IrankQ) 

{ 

int  j; 

for  (j=1;j  <=  n;]++)  lrank[indx[j]]=j; 


Appendix  5 
Test  data 

File:  Simul.000 
Hamming  Distance:  0 
Patterns  shown:  96 
Number  recognized:  10 
Number  not  recognized:  0 
Percent  recognized:  100 

File:  Simul.001 
Hamming  Distance:  1 
Patterns  shown:  96 

•  i  i  • 

Number  recognized:  9 
Number  not  recognized:  1 
Percent  recognized:  90 

File:  Simul.002 
Hamming  Distance:  2 
Patterns  shown:  96 
Number  recognized:  7 
Number  not  recognized:  3 
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Percent  recognized:  70 

File:  Simul.003 
Hamming  Distance:  3 
Patterns  shown:  96 
Number  recognized:  5 
Number  not  recognized:  5 
Percent  recognized:  50 

File:  Optic.OOO 

• 

Hamming  Distance:  0 
Patterns  shown:  144 
Number  recognized:  10 
Number  not  recognized:  0 
Percent  recognized:  90 

File:  Optic.OOl 
Hamming  Distance:  1 
Patterns  shown:  144 
Number  recognized:  7 
Number  not  recognized:  3 


Percent  recognized:  70 


File:  Optic.002 
Hamming  Distance:  2 
Patterns  shown:  144 
Number  recognized:  6 
Number  not  recognized:  4 
Percent  recognized:  60 

File:  Optic.003 
Hamming  Distance:  3 
Patterns  shown:  96 
Number  recognized:  3 
Number  not  recognized:  7 
Percent  recognized:  30 
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